-- Logs begin at Thu 2022-06-16 15:15:22 UTC, end at Thu 2022-06-16 16:34:12 UTC. -- Jun 16 15:56:58.351787 dv-cinder-837167-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 16 15:56:58.361340 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 16 15:56:58.362995 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 16 15:56:58 2022] *** Jun 16 15:56:58.362995 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 16 15:56:58.362995 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 16 15:56:58.362995 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: nodename: dv-cinder-837167-3-iscsi Jun 16 15:56:58.362995 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: machine: x86_64 Jun 16 15:56:58.362995 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: clock source: unix Jun 16 15:56:58.362995 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: pcre jit disabled Jun 16 15:56:58.362995 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: detected number of CPU cores: 4 Jun 16 15:56:58.362995 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: current working directory: / Jun 16 15:56:58.362995 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: detected binary path: /usr/bin/uwsgi-core Jun 16 15:56:58.362995 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: your processes number limit is 47680 Jun 16 15:56:58.362995 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: your memory page size is 4096 bytes Jun 16 15:56:58.362995 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: detected max file descriptor number: 2048 Jun 16 15:56:58.362995 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: lock engine: pthread robust mutexes Jun 16 15:56:58.364221 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: thunder lock: enabled Jun 16 15:56:58.364221 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 16 15:56:58.364221 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 16 15:56:58.410257 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: Python main interpreter initialized at 0x562b6a5159c0 Jun 16 15:56:58.410257 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: python threads support enabled Jun 16 15:56:58.410257 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: your server socket listen backlog is limited to 100 connections Jun 16 15:56:58.410257 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: your mercy for graceful operations on workers is 90 seconds Jun 16 15:56:58.410658 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: mapped 403077 bytes (393 KB) for 2 cores Jun 16 15:56:58.410734 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: *** Operational MODE: preforking *** Jun 16 15:56:58.410985 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: *** uWSGI is running in multiple interpreter mode *** Jun 16 15:56:58.410985 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: spawned uWSGI master process (pid: 80427) Jun 16 15:56:58.411203 dv-cinder-837167-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 16 15:56:58.412288 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: spawned uWSGI worker 1 (pid: 80439, cores: 1) Jun 16 15:56:58.412288 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: spawned uWSGI worker 2 (pid: 80440, cores: 1) Jun 16 15:56:58.412288 dv-cinder-837167-3-iscsi devstack@keystone.service[80427]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 16 15:57:04.137462 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 15:57:04.137462 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: from cryptography.utils import int_from_bytes Jun 16 15:57:04.138534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 15:57:04.138534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: from cryptography.utils import int_from_bytes Jun 16 15:57:04.190209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 15:57:04.190395 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 15:57:04.190589 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] command line args: [] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 15:57:04.190878 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 15:57:04.191056 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ================================================================================ {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 15:57:04.191318 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] admin_token = **** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.191573 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] config_dir = [] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.191770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.191959 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] config_source = [] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.192140 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] debug = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.192345 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.192855 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.192944 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.193158 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.193405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.193639 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] list_limit = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.193820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] log_config_append = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.194026 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.194211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] log_dir = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.194401 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] log_file = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.194605 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.194862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.195117 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.195372 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.195549 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.195732 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.195981 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.196230 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.196533 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.197267 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.197507 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.197745 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.198026 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.198280 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.198565 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.199093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.199267 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] publish_errors = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.199668 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.199802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.199959 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.200162 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.200392 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.200633 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] standard_threads = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.200875 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.201123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.201303 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.201484 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] use_journal = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.201664 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] use_json = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.201848 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] use_stderr = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.202059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] use_syslog = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.202300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.202568 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.202810 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.203076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.203320 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.203575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.203816 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.204070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.204309 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.204561 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.204867 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.205072 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.205315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.205566 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.205805 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.206095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.206348 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.206588 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.206849 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.207108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.207362 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.207600 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.207859 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.208128 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.208387 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.208644 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.208881 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.209124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.209350 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.209589 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.209843 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.210129 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.210361 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.210596 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.210853 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.211105 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.211347 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.211601 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.211843 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.212188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.212431 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.212666 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.212903 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.213138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.213373 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.213616 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.213867 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.214132 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.214373 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.214607 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.214848 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.215086 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.215339 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.215578 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.215816 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.216064 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.216297 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.216541 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.216778 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.217018 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.217274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.217513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.217752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.218008 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.218275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.218363 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 15:57:04.218363 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: from cryptography.utils import int_from_bytes Jun 16 15:57:04.218943 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.219026 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 15:57:04.219026 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: from cryptography.utils import int_from_bytes Jun 16 15:57:04.219152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.219152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.219288 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.219451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.219691 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.219944 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.220200 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.220438 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.220680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.220920 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.221434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.221692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.221977 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.222208 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.222450 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.222699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.222917 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.223154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.223385 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.223651 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.223895 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.224138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.224378 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.224623 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.224865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.225101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.225340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.225578 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.225809 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.226067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.226308 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.226652 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.226812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.227002 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.227257 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.227503 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.228399 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.228633 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.228879 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.229176 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] auth.external = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.229395 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.229637 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.229880 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.230217 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] auth.password = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.230475 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] auth.token = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.230703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.230943 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.231187 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.231425 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.231677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.231925 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.232160 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.232403 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] credential.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.232653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.232907 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.233152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.233392 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.233643 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.233875 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.234182 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.234422 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.234654 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.235002 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.235270 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.235439 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.235705 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.235952 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.236211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.236452 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] federation.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.236690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.236953 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.237173 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.237406 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.237656 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.237900 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.238181 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.238420 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.238679 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.238917 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.239169 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.239407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.239655 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.239904 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.240141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.240379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.240629 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.240864 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.241107 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.241354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.241610 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.241844 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.242104 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.242339 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.242592 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.242839 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.243091 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.243402 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.244141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.244558 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.244647 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.244878 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.245120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.245353 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.245595 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.245827 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.246114 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.246338 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.246596 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.246829 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.247086 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.247353 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.247571 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.247816 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.248067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.248332 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.248577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.248778 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.249048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.249395 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.249545 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.249735 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.249993 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.250248 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.250501 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.250735 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.251227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.251477 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.251726 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.251965 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.252201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.252438 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.252672 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.252916 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.253180 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.253413 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.253662 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.253909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.254197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.254436 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.254668 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.254908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.255146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.255386 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.255618 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.255868 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.256117 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.256361 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.256611 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.256859 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.257089 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.257341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.257580 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.257823 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.258214 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.258314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.258542 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.258813 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.259428 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.259690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.260139 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.260376 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.260627 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.260963 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.261109 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.261399 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.261580 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.261815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.262084 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.262366 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] resource.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.262569 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.262822 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.263073 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.263318 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.263576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.263810 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.264060 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.264311 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.264535 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.264772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] role.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.265019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] role.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.265253 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.265504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.265752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.266063 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.266318 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.266567 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.266820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.267217 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.267356 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.267542 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.267788 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.268046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.268294 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.268537 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.268968 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.269205 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.269461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.269714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.269990 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.270250 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.270488 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.270750 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.271005 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.271245 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.271487 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.271684 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 15:57:04.271831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.271963 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 15:57:04.272099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.272224 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] command line args: [] {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 15:57:04.272411 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.272534 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 15:57:04.272663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.272827 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ================================================================================ {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 15:57:04.273207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.273375 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] admin_token = **** {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.273557 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.273721 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] config_dir = [] {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.273888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.274228 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.274399 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.274691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] config_source = [] {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.274884 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] token.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.275021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] debug = True {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.275154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.275288 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.275481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.275660 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.275805 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.275976 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.276140 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.276457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.276611 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.276744 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.276916 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.277048 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] list_limit = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.277048 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] log_config_append = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.277242 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.277356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.277554 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.277634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] log_dir = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.277634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] log_file = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.277857 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.277857 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.278065 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.278144 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.278217 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.278290 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.278371 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.278447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.278528 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.278607 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.278680 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.278769 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.278862 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.278933 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.279033 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.279177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.279312 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.279446 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.connection = **** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.279596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.279697 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.279824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.279824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.280049 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.280133 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.280207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.280269 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.280328 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.280473 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.280573 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.280573 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.280772 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.280880 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.280880 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.281108 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.281199 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.281263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] publish_errors = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.281332 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.281405 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.281405 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.281528 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.281594 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.281669 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.281785 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.281955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.282088 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.282221 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.282346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] standard_threads = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.282426 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.282488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.282558 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.282626 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.282687 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.282756 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.282831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.283111 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] use_journal = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.283185 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 15:57:04.283249 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] use_json = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.283249 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] use_stderr = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.283249 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] use_syslog = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.283249 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:57:04.283249 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.283621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.283690 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.283879 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.284129 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.284373 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.284621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.284857 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.285107 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.285352 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.285603 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.285836 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.286108 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.286348 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.286592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.286841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.287076 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.287329 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.287582 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.287828 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.288063 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.288317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.288583 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.288835 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.289086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.289316 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.289545 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.289774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.290036 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.290835 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.291164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.291411 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.291644 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.291892 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.292139 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.292376 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.292624 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.292855 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.293090 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.293327 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.293557 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.293786 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.294043 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.294275 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.294518 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.294770 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.295000 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.295235 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.295463 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.295696 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.295929 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.296175 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.296403 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.296673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.296876 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.297336 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.297675 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.297989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.298192 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.298448 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.298683 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.299036 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.299140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.299401 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.299636 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.299866 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.300105 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.300336 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.300609 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.300847 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.301100 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.301347 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.301581 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.301813 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.302078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.302329 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.302565 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.302802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.303044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.303282 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.303516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.303750 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.303982 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.304216 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.304492 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.304790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.305128 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.305365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.305611 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.305900 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.306669 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.306906 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.307141 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.307372 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.307604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.307842 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.308072 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.308315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.308550 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.308806 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.309042 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.309301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.309559 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.309808 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.310062 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] auth.external = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.310299 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.310831 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.311194 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.311360 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] auth.password = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.311547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] auth.token = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.311783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.312036 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.312365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.312624 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.312869 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.313155 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.313371 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.313736 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] credential.caching = True {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.314321 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.314466 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.314617 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.314854 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.315117 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.315332 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.315746 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.316005 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.316273 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.316533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.316818 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.317065 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.317328 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.317576 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.317835 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.318099 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] federation.caching = True {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.318338 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.318584 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.318830 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.319058 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.319307 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.319543 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.319797 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.320028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.320285 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.320520 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.320769 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.321004 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] identity.caching = True {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.321247 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.321543 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.321779 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.322124 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.322435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.322668 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.322904 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.323152 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.323409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.323636 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.323868 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.324098 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.324348 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.324598 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.324842 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.325094 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.325338 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.325611 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.325848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.326109 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.326341 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.326576 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.326803 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.327036 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.327280 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.327518 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.327763 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.327992 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.328237 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.328482 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.328713 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.328955 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.329192 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.329426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.329661 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.329891 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.330153 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.330389 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.330636 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.330873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.331108 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.331355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.331599 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.331829 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.332205 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.332334 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.332525 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.332758 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.332992 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.333226 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.333455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.user = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.333699 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.333982 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.334215 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.334462 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.334705 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.334948 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.335177 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.335404 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.335637 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.335869 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.336102 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.336327 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.336574 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.336813 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.337142 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.337334 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.337754 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.338013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.338263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.338499 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.338727 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.338962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.339203 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.339446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.339691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.339927 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.340177 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.340403 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.340641 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.340879 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.341112 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.341345 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.341587 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.341827 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.342077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.342313 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.342558 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] resource.caching = True {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.342805 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.343053 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.343287 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.343542 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.343791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.344023 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.344267 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.344497 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.344736 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.344967 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] role.caching = True {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.345210 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] role.driver = sql {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.345436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.345683 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.345997 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.346196 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.346440 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.346683 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.346926 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.347167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.347565 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.347795 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.347928 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.348108 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.348353 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.348596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.349035 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.349268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.349514 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.349758 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.350034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.350283 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.350515 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.350770 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.351018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.351291 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.351480 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.351924 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.352173 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.352507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.353210 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.353460 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.353701 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.353953 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.354313 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.354445 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] token.caching = True {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.354630 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.354879 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.355111 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.355367 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.355611 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.355851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.356114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.356347 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.356797 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.357039 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.357353 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.357618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.357864 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.358140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.358369 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.358769 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.359067 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.359346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] database.connection = **** {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.359589 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.359958 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.360200 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.360439 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.360670 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.360909 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.361142 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.361516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.361660 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.361828 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.362097 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.362331 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.362578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.362810 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.363052 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.363279 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.363518 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.363752 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:57:04.363918 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 15:57:04.400179 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.400390 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.400527 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.410443 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.442167 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.442370 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.476985 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.477219 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.477442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.487297 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.501626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.508389 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.508610 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.548433 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.552313 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.552558 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.586745 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.586951 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.624551 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.624782 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.625014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.625524 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.625754 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.626251 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.632911 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.633133 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.633596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.633819 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.634104 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.634319 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.651279 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.651448 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.651641 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.651860 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.651945 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.652077 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.652162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.652292 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.652379 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.652505 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.652590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.652753 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.652965 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.653176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.664123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.664331 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.677860 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.678134 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.678247 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.678328 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.678403 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.678477 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.678550 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.678632 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.678706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.678821 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.678906 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.679035 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.679118 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.679251 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:57:04.696862 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.697141 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 15:57:04.697308 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.697487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.697641 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 15:57:04.697750 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 15:57:04.697846 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.697991 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.698111 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 15:57:04.698240 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.698393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.698526 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.698655 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.698826 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.698960 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.699101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.709820 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.710061 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.710192 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.710339 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.710469 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.710625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.710749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.710880 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.719719 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.719876 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.723030 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.723151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.723303 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.723421 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.723582 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.723726 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.723856 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.723945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.724130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.724214 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.724385 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.724523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.724632 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.724731 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.724899 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.724985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.725162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.725247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.725428 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.725511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.725689 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.725830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.753554 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.753748 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.755263 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.755355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.755548 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.755636 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.761209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.761363 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.761490 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.761632 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.761901 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.762002 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.769385 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.769656 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.769746 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.769973 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.774296 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.774504 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.775160 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.775419 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.775557 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.775779 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.775899 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.776055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.776277 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.776415 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.776543 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.776659 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.776820 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.776950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.777090 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.777358 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.797023 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.797227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.797317 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.797421 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.802029 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.802157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.805707 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.805829 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.806054 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.806270 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.811515 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.812600 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.815295 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.816305 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.819067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.819362 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.819647 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.819930 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.820101 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.820204 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.820409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.820493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.820685 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.820961 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.821232 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.821570 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.840865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.841159 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.841430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.841697 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.841899 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.842220 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.842485 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.842745 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.849828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.850773 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.851614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.851889 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.852522 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.852853 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.857323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.857614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.857889 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.858143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.858234 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.858437 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.858596 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.858712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.859007 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.859279 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.872134 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.872428 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.872708 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.872795 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.872992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.873089 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.873299 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.873382 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.873575 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.873658 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.873850 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.873956 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.874250 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.874522 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.892596 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.893108 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.895977 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.896437 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.899361 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.900397 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.901792 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.902206 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.903437 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.903716 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.903861 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.904012 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.904138 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.904414 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.911517 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.911791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.913177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.913431 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.917241 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.917476 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.917561 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.917767 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.917948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.918204 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.918289 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.918490 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.927602 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.927778 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.927913 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.928002 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.928232 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.928474 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.936794 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.937076 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.937227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:57:04.937353 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.937522 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.937628 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.937805 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.937896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.938108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.938342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.938434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.938616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.938762 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.938896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.939040 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.939169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.939304 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.939436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.939578 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.939699 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.939848 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.939971 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.940119 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.940231 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.940396 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.940477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.940664 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.940931 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:57:04.979958 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x562b6a5159c0 pid: 80439 (default app) Jun 16 15:57:04.980949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x562b6a5159c0 pid: 80440 (default app) Jun 16 15:57:04.983770 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60f05c-bae5-4675-9100-6a2611a2c8d5 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:04.984062 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60f05c-bae5-4675-9100-6a2611a2c8d5 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:04.984336 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60f05c-bae5-4675-9100-6a2611a2c8d5 None None] PATH_INFO: `/v3` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:04.988751 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Thu Jun 16 15:57:04 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:13.368197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ec5e5-699f-4f7d-bc71-a386986ccd55 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:13.368792 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ec5e5-699f-4f7d-bc71-a386986ccd55 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:13.368792 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ec5e5-699f-4f7d-bc71-a386986ccd55 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:13.373997 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1/2] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:13 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:57:13.381013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ad292-3e79-49f3-b267-b06300ad8fdb None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:13.381548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ad292-3e79-49f3-b267-b06300ad8fdb None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:13.381548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ad292-3e79-49f3-b267-b06300ad8fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:13.457171 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: INFO dbcounter [None req-c39ad292-3e79-49f3-b267-b06300ad8fdb None None] Registered counter for database keystone Jun 16 15:57:13.480209 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG oslo_db.sqlalchemy.engines [None req-c39ad292-3e79-49f3-b267-b06300ad8fdb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80439) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 15:57:13.488041 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writer thread running {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 15:57:13.689091 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG passlib.handlers.bcrypt [None req-c39ad292-3e79-49f3-b267-b06300ad8fdb None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80439) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 15:57:13.689425 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG passlib.handlers.bcrypt [None req-c39ad292-3e79-49f3-b267-b06300ad8fdb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80439) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 15:57:13.745617 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-c39ad292-3e79-49f3-b267-b06300ad8fdb None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:13.747303 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-c39ad292-3e79-49f3-b267-b06300ad8fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:13.876242 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 2/3] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:13 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 496 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 16 15:57:14.002804 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ec5e5-699f-4f7d-bc71-a386986ccd55 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:14.075132 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: INFO dbcounter [None req-2f9ec5e5-699f-4f7d-bc71-a386986ccd55 None None] Registered counter for database keystone Jun 16 15:57:14.095432 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG oslo_db.sqlalchemy.engines [None req-2f9ec5e5-699f-4f7d-bc71-a386986ccd55 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80440) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 15:57:14.101786 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writer thread running {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 15:57:14.228097 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ec5e5-699f-4f7d-bc71-a386986ccd55 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:14.232554 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa1092a-f7d0-4507-bb8b-9ecf71616d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:14.233231 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1092a-f7d0-4507-bb8b-9ecf71616d29 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:14.233520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1092a-f7d0-4507-bb8b-9ecf71616d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:14.233799 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1092a-f7d0-4507-bb8b-9ecf71616d29 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:14.242877 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa1092a-f7d0-4507-bb8b-9ecf71616d29 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:14.297351 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: WARNING keystone.common.rbac_enforcer.enforcer [None req-dfa1092a-f7d0-4507-bb8b-9ecf71616d29 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 15:57:14.297612 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa1092a-f7d0-4507-bb8b-9ecf71616d29 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:14.308861 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 2/4] 10.222.0.17 () {58 vars in 1324 bytes} [Thu Jun 16 15:57:14 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 15:57:14.314942 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ad292-3e79-49f3-b267-b06300ad8fdb None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:14.332905 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ad292-3e79-49f3-b267-b06300ad8fdb None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:14.337360 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072accb-2786-4fd6-b0f6-5be721494df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:14.338613 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072accb-2786-4fd6-b0f6-5be721494df9 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:14.339140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072accb-2786-4fd6-b0f6-5be721494df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:14.339140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072accb-2786-4fd6-b0f6-5be721494df9 admin admin] PATH_INFO: `/v3/projects/47ead53fd5cc424c8de9383e832d40a2` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:14.340526 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072accb-2786-4fd6-b0f6-5be721494df9 admin admin] RBAC: Authorizing `identity:get_project(project_id=47ead53fd5cc424c8de9383e832d40a2)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:14.391719 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f072accb-2786-4fd6-b0f6-5be721494df9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 15:57:14.392438 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072accb-2786-4fd6-b0f6-5be721494df9 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:14.396720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 3/5] 10.222.0.17 () {56 vars in 1207 bytes} [Thu Jun 16 15:57:14 2022] GET /identity/v3/projects/47ead53fd5cc424c8de9383e832d40a2 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:57:16.546481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0782fa4-f21c-45ee-92f9-f0e375dc9f40 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:16.547042 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0782fa4-f21c-45ee-92f9-f0e375dc9f40 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:16.547042 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0782fa4-f21c-45ee-92f9-f0e375dc9f40 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:16.551140 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 3/6] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:16.556858 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e656bdc-7491-4e9d-98fa-28900adaae36 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:16.557800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e656bdc-7491-4e9d-98fa-28900adaae36 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:16.557902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e656bdc-7491-4e9d-98fa-28900adaae36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:16.577620 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-6e656bdc-7491-4e9d-98fa-28900adaae36 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:16.579352 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-6e656bdc-7491-4e9d-98fa-28900adaae36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:16.587690 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 4/7] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 16 15:57:16.610977 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0782fa4-f21c-45ee-92f9-f0e375dc9f40 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:16.624143 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0782fa4-f21c-45ee-92f9-f0e375dc9f40 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:16.628828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a491b5-7231-43b5-a737-6f6fc6ff6edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:16.629477 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a491b5-7231-43b5-a737-6f6fc6ff6edf admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:16.629764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a491b5-7231-43b5-a737-6f6fc6ff6edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:16.630090 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a491b5-7231-43b5-a737-6f6fc6ff6edf admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:16.639254 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a491b5-7231-43b5-a737-6f6fc6ff6edf admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:16.641067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a491b5-7231-43b5-a737-6f6fc6ff6edf admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:16.650369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 4/8] 10.222.0.17 () {58 vars in 1324 bytes} [Thu Jun 16 15:57:16 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 15:57:16.656307 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e656bdc-7491-4e9d-98fa-28900adaae36 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:16.665632 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e656bdc-7491-4e9d-98fa-28900adaae36 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:16.670283 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392e323-6d90-4a41-977f-8553d2201eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:16.671160 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392e323-6d90-4a41-977f-8553d2201eac admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:16.671448 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392e323-6d90-4a41-977f-8553d2201eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:16.671714 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392e323-6d90-4a41-977f-8553d2201eac admin admin] PATH_INFO: `/v3/users/9593d0f94eed4e3e83f1323aab2716a7` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:16.673121 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392e323-6d90-4a41-977f-8553d2201eac admin admin] RBAC: Authorizing `identity:get_user(user_id=9593d0f94eed4e3e83f1323aab2716a7)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:16.675020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392e323-6d90-4a41-977f-8553d2201eac admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:16.676648 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 5/9] 10.222.0.17 () {56 vars in 1198 bytes} [Thu Jun 16 15:57:16 2022] GET /identity/v3/users/9593d0f94eed4e3e83f1323aab2716a7 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:18.945977 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5748a91-7fea-43e9-9c24-a43a3b04c608 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:18.946517 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5748a91-7fea-43e9-9c24-a43a3b04c608 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:18.946621 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5748a91-7fea-43e9-9c24-a43a3b04c608 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:18.947903 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 5/10] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:18.950643 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758d0a3-0efe-4f68-8b83-e07a5563bd09 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:18.951083 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758d0a3-0efe-4f68-8b83-e07a5563bd09 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:18.951214 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758d0a3-0efe-4f68-8b83-e07a5563bd09 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:18.952573 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 6/11] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:18.953077 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fec9c7-436a-48b7-9c59-4f7e2a8e59f4 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:18.953370 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fec9c7-436a-48b7-9c59-4f7e2a8e59f4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:18.953776 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fec9c7-436a-48b7-9c59-4f7e2a8e59f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:18.957374 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4e762-4a18-42d6-8734-d0d66811feb6 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:18.957662 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4e762-4a18-42d6-8734-d0d66811feb6 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:18.957971 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4e762-4a18-42d6-8734-d0d66811feb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:18.975305 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-03b4e762-4a18-42d6-8734-d0d66811feb6 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:18.976492 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-03b4e762-4a18-42d6-8734-d0d66811feb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:18.996470 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG passlib.handlers.bcrypt [None req-33fec9c7-436a-48b7-9c59-4f7e2a8e59f4 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80440) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 15:57:18.996898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG passlib.handlers.bcrypt [None req-33fec9c7-436a-48b7-9c59-4f7e2a8e59f4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80440) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 15:57:19.043669 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-33fec9c7-436a-48b7-9c59-4f7e2a8e59f4 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:19.045216 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-33fec9c7-436a-48b7-9c59-4f7e2a8e59f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:19.062214 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 7/12] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:57:19.084302 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4e762-4a18-42d6-8734-d0d66811feb6 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:19.109346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:19.150112 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 6/13] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:57:19.174276 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fec9c7-436a-48b7-9c59-4f7e2a8e59f4 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:19.191078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4e762-4a18-42d6-8734-d0d66811feb6 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:19.226451 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd136d9-9915-4e75-af35-044eefbe835b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:19.227114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd136d9-9915-4e75-af35-044eefbe835b None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:19.227498 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd136d9-9915-4e75-af35-044eefbe835b None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:19.227682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd136d9-9915-4e75-af35-044eefbe835b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:19.284802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fec9c7-436a-48b7-9c59-4f7e2a8e59f4 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:19.287314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:19.298664 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd136d9-9915-4e75-af35-044eefbe835b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:19.300394 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd136d9-9915-4e75-af35-044eefbe835b None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:19.323972 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:19.324757 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f1fd2-5f07-4b79-a80d-d78e3b569d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:19.325610 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f1fd2-5f07-4b79-a80d-d78e3b569d19 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:19.326015 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f1fd2-5f07-4b79-a80d-d78e3b569d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:19.326369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f1fd2-5f07-4b79-a80d-d78e3b569d19 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:19.340171 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f1fd2-5f07-4b79-a80d-d78e3b569d19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:57:19.351340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-802f1fd2-5f07-4b79-a80d-d78e3b569d19 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:57:19.351340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:19.351340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:19.351340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:19.351340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:19.351340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:19.351340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:19.351340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:19.351340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:19.351340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:19.351340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:19.351340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:19.351340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:57:19.351340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:57:19.351340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:57:19.351340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:57:19.352227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:57:19.352227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:57:19.352227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:57:19.352227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:57:19.352227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:19.352227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:19.352227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:19.352227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:19.352227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:19.352227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:19.352227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:19.352227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:19.352227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:19.352227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:19.352227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:19.352227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:19.353212 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:19.353212 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:19.353212 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:19.353212 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:19.353212 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:57:19.353212 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:57:19.353212 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:57:19.353212 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:57:19.353212 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:57:19.353212 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:57:19.353212 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:57:19.353212 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:57:19.353212 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 7/14] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 15:57:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:57:19.359148 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f1fd2-5f07-4b79-a80d-d78e3b569d19 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:19.402340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 8/15] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:57:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 15:57:19.408116 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd136d9-9915-4e75-af35-044eefbe835b None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:19.466579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f1fd2-5f07-4b79-a80d-d78e3b569d19 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:19.501940 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd239f3b-6e3f-4e05-85bd-769562fdf95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:19.502627 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd239f3b-6e3f-4e05-85bd-769562fdf95f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:19.503168 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd239f3b-6e3f-4e05-85bd-769562fdf95f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:19.503451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd239f3b-6e3f-4e05-85bd-769562fdf95f None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:19.504206 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd239f3b-6e3f-4e05-85bd-769562fdf95f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:19.504515 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd239f3b-6e3f-4e05-85bd-769562fdf95f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:19.505983 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd239f3b-6e3f-4e05-85bd-769562fdf95f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:19.509882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd136d9-9915-4e75-af35-044eefbe835b None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:19.520844 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 8/16] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:57:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:57:19.527135 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd239f3b-6e3f-4e05-85bd-769562fdf95f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:19.535664 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:19.549972 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3fe82-c6ca-47da-b536-3eae39a55a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:19.550707 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3fe82-c6ca-47da-b536-3eae39a55a9c None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:19.550991 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3fe82-c6ca-47da-b536-3eae39a55a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:19.551266 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3fe82-c6ca-47da-b536-3eae39a55a9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:19.556672 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:19.563086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e3fe82-c6ca-47da-b536-3eae39a55a9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:19.564618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e3fe82-c6ca-47da-b536-3eae39a55a9c None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:19.588368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-90e3fe82-c6ca-47da-b536-3eae39a55a9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:57:19.588368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:19.588368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:19.588368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:19.588368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:19.588368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:19.588368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:19.588368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:19.588368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:19.588368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:19.588368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:19.588368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:19.588368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:57:19.588368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:57:19.588368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:57:19.589590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:57:19.589590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:19.589590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:19.589590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:19.589590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:19.589590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:19.589590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:19.589590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:19.589590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:19.589590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:19.589590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:19.589590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:19.589590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:19.589590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:19.589590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:19.589590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:19.589590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:19.590620 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:57:19.590620 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:57:19.590620 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:57:19.590620 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:57:19.590620 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 9/17] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:57:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:57:19.595782 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3fe82-c6ca-47da-b536-3eae39a55a9c None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:19.641185 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd239f3b-6e3f-4e05-85bd-769562fdf95f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:19.675936 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e052b8-c238-41b0-8acc-f51b41fa5756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:19.676846 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e052b8-c238-41b0-8acc-f51b41fa5756 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:19.677251 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e052b8-c238-41b0-8acc-f51b41fa5756 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:19.677530 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e052b8-c238-41b0-8acc-f51b41fa5756 None admin] PATH_INFO: `/v3/users/9593d0f94eed4e3e83f1323aab2716a7` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:19.679216 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e052b8-c238-41b0-8acc-f51b41fa5756 None admin] RBAC: Authorizing `identity:get_user(user_id=9593d0f94eed4e3e83f1323aab2716a7)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:19.680706 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e052b8-c238-41b0-8acc-f51b41fa5756 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:19.682422 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 9/18] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:57:19 2022] GET /identity/v3/users/9593d0f94eed4e3e83f1323aab2716a7 => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:57:19.692256 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e052b8-c238-41b0-8acc-f51b41fa5756 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:19.708005 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3fe82-c6ca-47da-b536-3eae39a55a9c None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:19.743960 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e115a00-5462-4583-b6c7-8ea6cfac1027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:19.744716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e115a00-5462-4583-b6c7-8ea6cfac1027 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:19.745549 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e115a00-5462-4583-b6c7-8ea6cfac1027 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:19.746502 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e115a00-5462-4583-b6c7-8ea6cfac1027 None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:19.747239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e115a00-5462-4583-b6c7-8ea6cfac1027 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:19.747548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e115a00-5462-4583-b6c7-8ea6cfac1027 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:19.748941 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e115a00-5462-4583-b6c7-8ea6cfac1027 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:19.767505 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 10/19] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:57:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:57:19.793204 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:19.801633 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e052b8-c238-41b0-8acc-f51b41fa5756 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:19.840698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac0c27-96aa-4b7a-882b-bd45bca5e2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:19.841663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac0c27-96aa-4b7a-882b-bd45bca5e2e7 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:19.841976 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac0c27-96aa-4b7a-882b-bd45bca5e2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:19.842262 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac0c27-96aa-4b7a-882b-bd45bca5e2e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:19.843572 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ac0c27-96aa-4b7a-882b-bd45bca5e2e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:19.845097 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ac0c27-96aa-4b7a-882b-bd45bca5e2e7 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:19.846827 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 10/20] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:57:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:57:19.852959 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e115a00-5462-4583-b6c7-8ea6cfac1027 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:19.889756 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:19.962232 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e115a00-5462-4583-b6c7-8ea6cfac1027 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:20.000326 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022444f9-d5fd-4482-8229-373a59148360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:20.000990 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022444f9-d5fd-4482-8229-373a59148360 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:20.001278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022444f9-d5fd-4482-8229-373a59148360 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:20.001552 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022444f9-d5fd-4482-8229-373a59148360 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:20.002426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022444f9-d5fd-4482-8229-373a59148360 None admin] RBAC: Adding query filter params (role.id=bbbc77ec645d4ff5a8bf29ce0377f578, scope.domain.id=default, user.id=9593d0f94eed4e3e83f1323aab2716a7) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:20.002982 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022444f9-d5fd-4482-8229-373a59148360 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:20.004465 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022444f9-d5fd-4482-8229-373a59148360 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:20.012881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 11/21] 10.222.0.17 () {56 vars in 1322 bytes} [Thu Jun 16 15:57:19 2022] GET /identity/v3/role_assignments?user.id=9593d0f94eed4e3e83f1323aab2716a7&scope.domain.id=default&role.id=bbbc77ec645d4ff5a8bf29ce0377f578 => generated 233 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:22.630106 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbd5d9-3330-4a95-b4cf-0078c14677c9 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:22.631216 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbd5d9-3330-4a95-b4cf-0078c14677c9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:22.631707 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbd5d9-3330-4a95-b4cf-0078c14677c9 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:22.633676 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 11/22] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:22.639714 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89acb54-ff32-4ef3-8b3a-950685b858f6 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:22.640599 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89acb54-ff32-4ef3-8b3a-950685b858f6 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:22.641103 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89acb54-ff32-4ef3-8b3a-950685b858f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:22.664346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-b89acb54-ff32-4ef3-8b3a-950685b858f6 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:22.665779 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-b89acb54-ff32-4ef3-8b3a-950685b858f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:22.752655 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b861d-724f-42c8-b87a-7ef4880234c9 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:22.753534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b861d-724f-42c8-b87a-7ef4880234c9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:22.754085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b861d-724f-42c8-b87a-7ef4880234c9 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:22.756148 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 12/23] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:22.762161 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c10c14-6853-4915-ba9a-12c12674c722 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:22.762936 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c10c14-6853-4915-ba9a-12c12674c722 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:22.763421 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c10c14-6853-4915-ba9a-12c12674c722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:22.771838 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 12/24] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:22.781421 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d381c-e0d8-4899-b1ae-4f00154a98ff None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:22.781712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d381c-e0d8-4899-b1ae-4f00154a98ff None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:22.782187 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d381c-e0d8-4899-b1ae-4f00154a98ff None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:22.783801 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 13/25] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:57:22.793327 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204e23f-7a54-4a62-976a-d508c8981c37 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:22.793905 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204e23f-7a54-4a62-976a-d508c8981c37 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:22.794514 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204e23f-7a54-4a62-976a-d508c8981c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:22.803248 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-06c10c14-6853-4915-ba9a-12c12674c722 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:22.804815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-06c10c14-6853-4915-ba9a-12c12674c722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:22.825856 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-5204e23f-7a54-4a62-976a-d508c8981c37 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:22.827128 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-5204e23f-7a54-4a62-976a-d508c8981c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:22.887318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:22.936874 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 13/26] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:22.939119 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 14/27] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:57:22.939243 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c10c14-6853-4915-ba9a-12c12674c722 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:22.974456 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204e23f-7a54-4a62-976a-d508c8981c37 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:23.095136 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:23.108490 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c10c14-6853-4915-ba9a-12c12674c722 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:23.183447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093fd1f-2852-47cd-af17-045a3e8f1f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:23.184122 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093fd1f-2852-47cd-af17-045a3e8f1f81 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:23.184607 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093fd1f-2852-47cd-af17-045a3e8f1f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:23.184892 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093fd1f-2852-47cd-af17-045a3e8f1f81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:23.190774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204e23f-7a54-4a62-976a-d508c8981c37 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:23.209576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2093fd1f-2852-47cd-af17-045a3e8f1f81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:57:23.219224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-2093fd1f-2852-47cd-af17-045a3e8f1f81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:57:23.219224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:23.219224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:23.219224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:23.219224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:23.219224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:23.219224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:23.219224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:23.219224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:23.219224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:23.219224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:23.219224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:23.219224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:57:23.219224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:57:23.219224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:57:23.219224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:57:23.220124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:57:23.220124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:57:23.220124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:57:23.220124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:57:23.220124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:23.220124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:23.220124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:23.220124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:23.220124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:23.220124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:23.220124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:23.220124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:23.220124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:23.220124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:23.220124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:23.220124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:23.221016 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:23.221016 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:23.221016 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:23.221016 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:23.221016 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:57:23.221016 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:57:23.221016 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:57:23.221016 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:57:23.221016 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:57:23.221016 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:57:23.221016 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:57:23.221016 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:57:23.223297 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 14/28] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 15:57:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:57:23.224276 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093fd1f-2852-47cd-af17-045a3e8f1f81 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:23.249736 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea9a0f-589e-43cf-934b-46f115f83e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:23.250739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea9a0f-589e-43cf-934b-46f115f83e0a None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:23.251127 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea9a0f-589e-43cf-934b-46f115f83e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:23.251510 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea9a0f-589e-43cf-934b-46f115f83e0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:23.263419 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea9a0f-589e-43cf-934b-46f115f83e0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:23.265075 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea9a0f-589e-43cf-934b-46f115f83e0a None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:23.283895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-14ea9a0f-589e-43cf-934b-46f115f83e0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:57:23.283895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:23.283895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:23.283895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:23.283895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:23.283895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:23.283895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:23.283895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:23.283895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:23.283895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:23.283895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:23.283895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:23.283895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:57:23.283895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:57:23.283895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:57:23.284776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:57:23.284776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:23.284776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:23.284776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:23.284776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:23.284776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:23.284776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:23.284776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:23.284776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:23.284776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:23.284776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:23.284776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:23.284776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:23.284776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:23.284776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:23.284776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:23.284776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:23.285665 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:57:23.285665 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:57:23.285665 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:57:23.285665 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:57:23.285987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 15/29] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:57:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:57:23.287670 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3944ddd-7fa0-4a7d-8eef-ebd58dd23e88 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:23.288201 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3944ddd-7fa0-4a7d-8eef-ebd58dd23e88 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:23.288583 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3944ddd-7fa0-4a7d-8eef-ebd58dd23e88 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:23.290021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 16/30] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:23.291017 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3944ddd-7fa0-4a7d-8eef-ebd58dd23e88 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:23.310489 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:23.397293 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093fd1f-2852-47cd-af17-045a3e8f1f81 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:23.424048 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3944ddd-7fa0-4a7d-8eef-ebd58dd23e88 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:23.442235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1e53d-bfe7-4bb3-8aed-4e877455a795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:23.442840 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1e53d-bfe7-4bb3-8aed-4e877455a795 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:23.443284 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1e53d-bfe7-4bb3-8aed-4e877455a795 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:23.443563 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1e53d-bfe7-4bb3-8aed-4e877455a795 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:23.444432 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c1e53d-bfe7-4bb3-8aed-4e877455a795 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:23.446083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c1e53d-bfe7-4bb3-8aed-4e877455a795 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:23.469273 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff041541-47a6-43c2-8bb2-e579e809dd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:23.470140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff041541-47a6-43c2-8bb2-e579e809dd57 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:23.470529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff041541-47a6-43c2-8bb2-e579e809dd57 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:23.470914 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff041541-47a6-43c2-8bb2-e579e809dd57 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:23.471793 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff041541-47a6-43c2-8bb2-e579e809dd57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:23.472209 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff041541-47a6-43c2-8bb2-e579e809dd57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:23.473869 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff041541-47a6-43c2-8bb2-e579e809dd57 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:23.488648 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 17/31] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:57:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:23.489962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff041541-47a6-43c2-8bb2-e579e809dd57 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:23.572476 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:23.593244 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 15/32] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jun 16 15:57:23 2022] POST /identity/v3/roles => generated 225 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:57:23.599371 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7602b-5b54-4b16-9e61-ef75406091be None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:23.599689 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7602b-5b54-4b16-9e61-ef75406091be None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:23.599945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7602b-5b54-4b16-9e61-ef75406091be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:23.611035 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff041541-47a6-43c2-8bb2-e579e809dd57 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:23.636506 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-70a7602b-5b54-4b16-9e61-ef75406091be None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:23.638412 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-70a7602b-5b54-4b16-9e61-ef75406091be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:23.651368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d0d768-c8e3-4ab3-8ce6-d4de3258051c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:23.652597 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0d768-c8e3-4ab3-8ce6-d4de3258051c None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:23.657983 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0d768-c8e3-4ab3-8ce6-d4de3258051c None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:23.657983 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0d768-c8e3-4ab3-8ce6-d4de3258051c None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:23.658347 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:23.658506 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d0d768-c8e3-4ab3-8ce6-d4de3258051c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:23.658506 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d0d768-c8e3-4ab3-8ce6-d4de3258051c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:23.658506 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d0d768-c8e3-4ab3-8ce6-d4de3258051c None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:23.669648 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 18/33] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:57:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:23.670639 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d0d768-c8e3-4ab3-8ce6-d4de3258051c None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:23.779255 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 16/34] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:23.780075 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7602b-5b54-4b16-9e61-ef75406091be None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:23.813028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d0d768-c8e3-4ab3-8ce6-d4de3258051c None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:23.910455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:23.934293 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45941263-f33a-4e47-b6e2-d77ad24adc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:23.935471 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45941263-f33a-4e47-b6e2-d77ad24adc46 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:23.935901 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45941263-f33a-4e47-b6e2-d77ad24adc46 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:23.936321 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45941263-f33a-4e47-b6e2-d77ad24adc46 None admin] PATH_INFO: `/v3/users/9593d0f94eed4e3e83f1323aab2716a7` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:23.939694 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45941263-f33a-4e47-b6e2-d77ad24adc46 None admin] RBAC: Authorizing `identity:get_user(user_id=9593d0f94eed4e3e83f1323aab2716a7)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:23.941780 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45941263-f33a-4e47-b6e2-d77ad24adc46 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:23.944054 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 19/35] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:57:23 2022] GET /identity/v3/users/9593d0f94eed4e3e83f1323aab2716a7 => generated 247 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:23.945048 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45941263-f33a-4e47-b6e2-d77ad24adc46 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:24.051918 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7602b-5b54-4b16-9e61-ef75406091be None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:24.096918 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45941263-f33a-4e47-b6e2-d77ad24adc46 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:24.112833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a7a3c-7748-4b1e-ac64-2e4a0c63c33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:24.113538 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a7a3c-7748-4b1e-ac64-2e4a0c63c33b None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:24.113820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a7a3c-7748-4b1e-ac64-2e4a0c63c33b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:24.114277 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a7a3c-7748-4b1e-ac64-2e4a0c63c33b None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:24.115046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884a7a3c-7748-4b1e-ac64-2e4a0c63c33b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:24.116925 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884a7a3c-7748-4b1e-ac64-2e4a0c63c33b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:24.143926 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:24.144111 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd30abf-ab50-4a77-867b-060a02ff23dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:24.145190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd30abf-ab50-4a77-867b-060a02ff23dd None admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:24.145712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd30abf-ab50-4a77-867b-060a02ff23dd None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:24.147151 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd30abf-ab50-4a77-867b-060a02ff23dd None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:24.148870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd30abf-ab50-4a77-867b-060a02ff23dd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:24.152317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd30abf-ab50-4a77-867b-060a02ff23dd None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:24.165066 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 17/36] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jun 16 15:57:23 2022] POST /identity/v3/projects => generated 301 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:57:24.170065 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a7a3c-7748-4b1e-ac64-2e4a0c63c33b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:24.306310 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a7a3c-7748-4b1e-ac64-2e4a0c63c33b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:24.359907 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571dc427-e9a8-41cf-9844-9bd4c643d816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:24.360723 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571dc427-e9a8-41cf-9844-9bd4c643d816 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:24.361012 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571dc427-e9a8-41cf-9844-9bd4c643d816 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:24.361293 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571dc427-e9a8-41cf-9844-9bd4c643d816 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:24.362556 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571dc427-e9a8-41cf-9844-9bd4c643d816 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:24.364053 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571dc427-e9a8-41cf-9844-9bd4c643d816 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:24.365762 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 18/37] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:57:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:24.367206 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62c7bc-6130-4bba-877b-1fb6be6b5b7b None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:24.367502 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62c7bc-6130-4bba-877b-1fb6be6b5b7b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:24.367782 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62c7bc-6130-4bba-877b-1fb6be6b5b7b None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:24.369319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 19/38] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:24.371081 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e47bb-d392-46af-8df1-e69fa11904d8 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:24.371691 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e47bb-d392-46af-8df1-e69fa11904d8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:24.371844 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e47bb-d392-46af-8df1-e69fa11904d8 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:24.373879 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 20/39] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:24.374654 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e47bb-d392-46af-8df1-e69fa11904d8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:24.431817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 20/40] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jun 16 15:57:23 2022] POST /identity/v3/roles => generated 219 bytes in 487 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:57:24.434077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b909f29-1759-4650-b3b6-441a9434334e None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:24.434375 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b909f29-1759-4650-b3b6-441a9434334e None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:24.434660 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b909f29-1759-4650-b3b6-441a9434334e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:24.457023 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-7b909f29-1759-4650-b3b6-441a9434334e None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:24.459069 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-7b909f29-1759-4650-b3b6-441a9434334e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:24.541045 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:24.560147 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 21/41] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:24.562177 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6852af-9c0f-444b-a494-03cc9f60fbb5 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:24.562466 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6852af-9c0f-444b-a494-03cc9f60fbb5 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:24.562748 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6852af-9c0f-444b-a494-03cc9f60fbb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:24.575152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:24.595488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-af6852af-9c0f-444b-a494-03cc9f60fbb5 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:24.597367 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-af6852af-9c0f-444b-a494-03cc9f60fbb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:24.626207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e47bb-d392-46af-8df1-e69fa11904d8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:24.697679 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d75674-7ebc-4688-ac9f-7632d52c6502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:24.699937 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d75674-7ebc-4688-ac9f-7632d52c6502 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:24.700451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d75674-7ebc-4688-ac9f-7632d52c6502 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:24.700928 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d75674-7ebc-4688-ac9f-7632d52c6502 None admin] PATH_INFO: `/v3/domains/default/users/9593d0f94eed4e3e83f1323aab2716a7/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:24.702659 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 22/42] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:24.703631 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6852af-9c0f-444b-a494-03cc9f60fbb5 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:24.711249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d75674-7ebc-4688-ac9f-7632d52c6502 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=9593d0f94eed4e3e83f1323aab2716a7, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:24.712753 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d75674-7ebc-4688-ac9f-7632d52c6502 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:24.733293 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 21/43] 10.222.0.17 () {58 vars in 1361 bytes} [Thu Jun 16 15:57:24 2022] PUT /identity/v3/domains/default/users/9593d0f94eed4e3e83f1323aab2716a7/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:57:24.734178 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d75674-7ebc-4688-ac9f-7632d52c6502 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:24.845468 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6852af-9c0f-444b-a494-03cc9f60fbb5 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:24.872935 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:24.875552 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d75674-7ebc-4688-ac9f-7632d52c6502 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:24.891254 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffd3e6-6a6b-4e4d-a469-bcda80fb3e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:24.891958 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffd3e6-6a6b-4e4d-a469-bcda80fb3e0c None admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:24.892238 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffd3e6-6a6b-4e4d-a469-bcda80fb3e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:24.892514 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffd3e6-6a6b-4e4d-a469-bcda80fb3e0c None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:24.893396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcffd3e6-6a6b-4e4d-a469-bcda80fb3e0c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:24.895078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcffd3e6-6a6b-4e4d-a469-bcda80fb3e0c None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:24.904289 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:24.911792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 23/44] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jun 16 15:57:24 2022] POST /identity/v3/roles => generated 223 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:57:24.924241 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325385b-f162-400d-953f-f9125422838c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:24.925176 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325385b-f162-400d-953f-f9125422838c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:24.925469 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325385b-f162-400d-953f-f9125422838c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:24.925753 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325385b-f162-400d-953f-f9125422838c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:24.927848 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5325385b-f162-400d-953f-f9125422838c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:24.929484 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5325385b-f162-400d-953f-f9125422838c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:24.931557 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 22/45] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:57:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:24.938458 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffd3e6-6a6b-4e4d-a469-bcda80fb3e0c None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:25.083311 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffd3e6-6a6b-4e4d-a469-bcda80fb3e0c None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:25.122603 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c7a75-41c0-4c09-9d9d-d3a32067e9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:25.123213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c7a75-41c0-4c09-9d9d-d3a32067e9b3 None admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:25.123501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c7a75-41c0-4c09-9d9d-d3a32067e9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:25.123785 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c7a75-41c0-4c09-9d9d-d3a32067e9b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:25.124511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c7a75-41c0-4c09-9d9d-d3a32067e9b3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:25.126033 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c7a75-41c0-4c09-9d9d-d3a32067e9b3 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:25.157267 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 24/46] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jun 16 15:57:24 2022] POST /identity/v3/projects => generated 312 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:57:26.891676 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c178aa-6ebd-4c62-919b-0170cdcbf675 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:26.892185 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c178aa-6ebd-4c62-919b-0170cdcbf675 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:26.892351 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c178aa-6ebd-4c62-919b-0170cdcbf675 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:26.893844 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 23/47] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:26.899136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30edbcc6-bb9d-41a2-9c2d-c472f250a89c None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:26.899559 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30edbcc6-bb9d-41a2-9c2d-c472f250a89c None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:26.899821 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30edbcc6-bb9d-41a2-9c2d-c472f250a89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:26.916414 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-30edbcc6-bb9d-41a2-9c2d-c472f250a89c None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:26.918156 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-30edbcc6-bb9d-41a2-9c2d-c472f250a89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:26.944387 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:26.988933 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 25/48] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:27.013157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c178aa-6ebd-4c62-919b-0170cdcbf675 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:27.117136 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c178aa-6ebd-4c62-919b-0170cdcbf675 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:27.156347 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967486d-40cd-4ad3-abd4-fcf7432edbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:27.157157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967486d-40cd-4ad3-abd4-fcf7432edbb2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:27.157446 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967486d-40cd-4ad3-abd4-fcf7432edbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:27.157726 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967486d-40cd-4ad3-abd4-fcf7432edbb2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:27.169610 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b967486d-40cd-4ad3-abd4-fcf7432edbb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:57:27.177394 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-b967486d-40cd-4ad3-abd4-fcf7432edbb2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:57:27.177394 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:27.177394 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:27.177394 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:27.177394 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:27.177394 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:27.177394 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:27.177394 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:27.177394 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:27.177394 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:27.177394 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:27.177394 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:27.177394 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:57:27.177394 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:57:27.177394 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:57:27.177394 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:57:27.178341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:57:27.178341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:57:27.178341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:57:27.178341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:57:27.178341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:27.178341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:27.178341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:27.178341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:27.178341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:27.178341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:27.178341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:27.178341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:27.178341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:27.178341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:27.178341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:27.178341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:27.179117 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:27.179117 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:27.179117 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:27.179117 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:27.179117 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:57:27.179117 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:57:27.179117 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:57:27.179117 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:57:27.179117 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:57:27.179117 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:57:27.179117 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:57:27.179117 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:57:27.179117 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 24/49] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 15:57:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:57:27.184908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30edbcc6-bb9d-41a2-9c2d-c472f250a89c None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:27.297097 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30edbcc6-bb9d-41a2-9c2d-c472f250a89c None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:27.321234 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763cbc91-4ad7-4444-9393-22a0daa43a2b None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:27.321523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763cbc91-4ad7-4444-9393-22a0daa43a2b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:27.321840 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763cbc91-4ad7-4444-9393-22a0daa43a2b None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:27.323275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 25/50] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:57:27.329739 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab01374-825c-4f46-b3d6-bfe796b362d6 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:27.330073 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab01374-825c-4f46-b3d6-bfe796b362d6 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:27.330355 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab01374-825c-4f46-b3d6-bfe796b362d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:27.336086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf4853-a5f6-40ae-8f72-c2267b9e5bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:27.336776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf4853-a5f6-40ae-8f72-c2267b9e5bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:27.337063 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf4853-a5f6-40ae-8f72-c2267b9e5bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:27.337341 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf4853-a5f6-40ae-8f72-c2267b9e5bf3 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:27.338136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf4853-a5f6-40ae-8f72-c2267b9e5bf3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:27.338451 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf4853-a5f6-40ae-8f72-c2267b9e5bf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:27.340174 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf4853-a5f6-40ae-8f72-c2267b9e5bf3 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:27.349134 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 26/51] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:57:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:27.351011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-dab01374-825c-4f46-b3d6-bfe796b362d6 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:27.352232 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-dab01374-825c-4f46-b3d6-bfe796b362d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:27.354759 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf4853-a5f6-40ae-8f72-c2267b9e5bf3 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:27.382479 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:27.402794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:27.426825 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 26/52] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:57:27.451189 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab01374-825c-4f46-b3d6-bfe796b362d6 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:27.466272 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf4853-a5f6-40ae-8f72-c2267b9e5bf3 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:27.504884 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc9a513-e010-4455-8c27-60e3f090c4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:27.505758 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9a513-e010-4455-8c27-60e3f090c4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:27.506270 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9a513-e010-4455-8c27-60e3f090c4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:27.506352 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9a513-e010-4455-8c27-60e3f090c4b3 None admin] PATH_INFO: `/v3/users/9593d0f94eed4e3e83f1323aab2716a7` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:27.507592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc9a513-e010-4455-8c27-60e3f090c4b3 None admin] RBAC: Authorizing `identity:get_user(user_id=9593d0f94eed4e3e83f1323aab2716a7)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:27.509043 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc9a513-e010-4455-8c27-60e3f090c4b3 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:27.510752 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 27/53] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:57:27 2022] GET /identity/v3/users/9593d0f94eed4e3e83f1323aab2716a7 => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:57:27.516268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc9a513-e010-4455-8c27-60e3f090c4b3 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:27.557417 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab01374-825c-4f46-b3d6-bfe796b362d6 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:27.594231 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151d6ac-4781-421f-9a9c-2c48b7fc9651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:27.595040 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151d6ac-4781-421f-9a9c-2c48b7fc9651 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:27.595324 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151d6ac-4781-421f-9a9c-2c48b7fc9651 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:27.595600 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151d6ac-4781-421f-9a9c-2c48b7fc9651 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:27.596804 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3151d6ac-4781-421f-9a9c-2c48b7fc9651 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:27.598338 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3151d6ac-4781-421f-9a9c-2c48b7fc9651 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:27.600016 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 27/54] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:57:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:57:27.606283 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151d6ac-4781-421f-9a9c-2c48b7fc9651 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:27.620308 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:27.631251 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc9a513-e010-4455-8c27-60e3f090c4b3 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:27.658820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:27.669552 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b5db3-adf7-40d8-8ac2-f3252521785a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:27.670402 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b5db3-adf7-40d8-8ac2-f3252521785a None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:27.670691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b5db3-adf7-40d8-8ac2-f3252521785a None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:27.670960 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b5db3-adf7-40d8-8ac2-f3252521785a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:27.672173 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20b5db3-adf7-40d8-8ac2-f3252521785a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:27.673636 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20b5db3-adf7-40d8-8ac2-f3252521785a None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:27.675325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 28/55] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:57:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:57:27.681101 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b5db3-adf7-40d8-8ac2-f3252521785a None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:27.715669 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151d6ac-4781-421f-9a9c-2c48b7fc9651 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:27.755014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b842a382-8a67-4a1c-a3d9-23b575bafa93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:27.755691 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842a382-8a67-4a1c-a3d9-23b575bafa93 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:27.755969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842a382-8a67-4a1c-a3d9-23b575bafa93 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:27.756243 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842a382-8a67-4a1c-a3d9-23b575bafa93 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:27.756972 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b842a382-8a67-4a1c-a3d9-23b575bafa93 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:27.758462 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b842a382-8a67-4a1c-a3d9-23b575bafa93 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:27.785676 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 28/56] 10.222.0.17 () {60 vars in 1140 bytes} [Thu Jun 16 15:57:27 2022] POST /identity/v3/projects => generated 298 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:57:27.806384 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b5db3-adf7-40d8-8ac2-f3252521785a None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:27.845387 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1163c-93cc-4360-a95d-3820b96e3833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:27.846118 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1163c-93cc-4360-a95d-3820b96e3833 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:27.846412 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1163c-93cc-4360-a95d-3820b96e3833 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:27.846695 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1163c-93cc-4360-a95d-3820b96e3833 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:27.847624 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1163c-93cc-4360-a95d-3820b96e3833 None admin] RBAC: Adding query filter params (role.id=bbbc77ec645d4ff5a8bf29ce0377f578, scope.domain.id=default, user.id=9593d0f94eed4e3e83f1323aab2716a7) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:27.847951 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1163c-93cc-4360-a95d-3820b96e3833 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:27.849480 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1163c-93cc-4360-a95d-3820b96e3833 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:27.857188 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 29/57] 10.222.0.17 () {56 vars in 1322 bytes} [Thu Jun 16 15:57:27 2022] GET /identity/v3/role_assignments?user.id=9593d0f94eed4e3e83f1323aab2716a7&scope.domain.id=default&role.id=bbbc77ec645d4ff5a8bf29ce0377f578 => generated 530 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:57:29.996687 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad22be-7b1f-4ea0-bae9-0c9d1fa043ca None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:29.997197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad22be-7b1f-4ea0-bae9-0c9d1fa043ca None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:29.997304 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad22be-7b1f-4ea0-bae9-0c9d1fa043ca None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:29.998668 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 29/58] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:30.004240 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df310fdd-078e-43fd-9641-6678bd6b502f None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:30.004644 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df310fdd-078e-43fd-9641-6678bd6b502f None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:30.004803 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df310fdd-078e-43fd-9641-6678bd6b502f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:30.022423 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-df310fdd-078e-43fd-9641-6678bd6b502f None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:30.023622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-df310fdd-078e-43fd-9641-6678bd6b502f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:30.042036 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:30.093993 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 30/59] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:30.118127 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad22be-7b1f-4ea0-bae9-0c9d1fa043ca None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:30.218690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:30.223187 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad22be-7b1f-4ea0-bae9-0c9d1fa043ca None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:30.264987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb60c56f-fff9-4492-85c1-87dcd0615539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:30.265790 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60c56f-fff9-4492-85c1-87dcd0615539 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:30.266112 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60c56f-fff9-4492-85c1-87dcd0615539 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:30.266395 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60c56f-fff9-4492-85c1-87dcd0615539 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:30.267578 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb60c56f-fff9-4492-85c1-87dcd0615539 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:30.269057 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb60c56f-fff9-4492-85c1-87dcd0615539 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:30.270738 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 30/60] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:57:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:30.276280 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df310fdd-078e-43fd-9641-6678bd6b502f None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:30.374932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df310fdd-078e-43fd-9641-6678bd6b502f None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:30.408727 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4821c3-007a-4dd3-b403-bf495fcd28a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:30.409399 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4821c3-007a-4dd3-b403-bf495fcd28a5 None admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:30.409686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4821c3-007a-4dd3-b403-bf495fcd28a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:30.409995 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4821c3-007a-4dd3-b403-bf495fcd28a5 None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:30.410702 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4821c3-007a-4dd3-b403-bf495fcd28a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:30.412177 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4821c3-007a-4dd3-b403-bf495fcd28a5 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:30.438440 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 31/61] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 15:57:30 2022] POST /identity/v3/users => generated 275 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:57:33.881858 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213036f-ff7b-4b26-8d85-94250fed65ac None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:33.882721 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213036f-ff7b-4b26-8d85-94250fed65ac None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:33.883142 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213036f-ff7b-4b26-8d85-94250fed65ac None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:33.884975 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 31/62] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:33.891956 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94827ab3-0652-4856-9c11-a03a18215161 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:33.892471 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94827ab3-0652-4856-9c11-a03a18215161 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:33.892544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94827ab3-0652-4856-9c11-a03a18215161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:33.917527 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-94827ab3-0652-4856-9c11-a03a18215161 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:33.923063 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-94827ab3-0652-4856-9c11-a03a18215161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:33.930292 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:33.974366 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f3dc6-c504-43bd-8add-4baa2fde3065 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:33.974888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f3dc6-c504-43bd-8add-4baa2fde3065 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:33.975311 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f3dc6-c504-43bd-8add-4baa2fde3065 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:33.976897 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 32/63] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:34.010214 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0481a8-ee63-4be9-9206-28c9e1dc4880 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:34.010728 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0481a8-ee63-4be9-9206-28c9e1dc4880 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:34.011148 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0481a8-ee63-4be9-9206-28c9e1dc4880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:34.031043 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 32/64] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:34.032384 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd274b8-0411-48ba-b21e-c33cf254bc04 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:34.032679 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd274b8-0411-48ba-b21e-c33cf254bc04 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:34.033431 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd274b8-0411-48ba-b21e-c33cf254bc04 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:34.034764 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 33/65] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:34.048426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b49de1-79b3-40e6-9fd7-5db4b1af463b None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:34.048721 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b49de1-79b3-40e6-9fd7-5db4b1af463b None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:34.048921 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b49de1-79b3-40e6-9fd7-5db4b1af463b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:34.051306 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-3e0481a8-ee63-4be9-9206-28c9e1dc4880 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:34.052532 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3e0481a8-ee63-4be9-9206-28c9e1dc4880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:34.075989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-60b49de1-79b3-40e6-9fd7-5db4b1af463b None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:34.078032 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-60b49de1-79b3-40e6-9fd7-5db4b1af463b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:34.162286 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 33/66] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:57:34.164232 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0481a8-ee63-4be9-9206-28c9e1dc4880 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:34.196560 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 34/67] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:57:34.201975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b49de1-79b3-40e6-9fd7-5db4b1af463b None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:34.306539 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:34.311448 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0481a8-ee63-4be9-9206-28c9e1dc4880 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:34.312775 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:34.355281 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96633419-76bb-4152-898b-fa8f635c628a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:34.356104 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96633419-76bb-4152-898b-fa8f635c628a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:34.356388 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96633419-76bb-4152-898b-fa8f635c628a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:34.356666 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96633419-76bb-4152-898b-fa8f635c628a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:34.357357 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b49de1-79b3-40e6-9fd7-5db4b1af463b None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:34.367750 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96633419-76bb-4152-898b-fa8f635c628a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:57:34.375948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-96633419-76bb-4152-898b-fa8f635c628a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:57:34.375948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:34.375948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:34.375948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:34.375948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:34.375948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:34.375948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:34.375948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:34.375948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:34.375948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:34.375948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:34.375948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:34.375948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:57:34.375948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:57:34.375948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:57:34.375948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:57:34.376862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:57:34.376862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:57:34.376862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:57:34.376862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:57:34.376862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:34.376862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:34.376862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:34.376862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:34.376862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:34.376862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:34.376862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:34.376862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:34.376862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:34.376862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:34.376862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:34.376862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:34.377615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:34.377615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:34.377615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:34.377615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:34.377615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:57:34.377615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:57:34.377615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:57:34.377615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:57:34.377615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:57:34.377615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:57:34.377615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:57:34.377615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:57:34.377615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 34/68] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:57:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:57:34.378276 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96633419-76bb-4152-898b-fa8f635c628a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:34.393960 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24633741-6b3e-46a0-a4d3-d6744640d144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:34.394774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24633741-6b3e-46a0-a4d3-d6744640d144 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:34.395064 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24633741-6b3e-46a0-a4d3-d6744640d144 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:34.395487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24633741-6b3e-46a0-a4d3-d6744640d144 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:34.407622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24633741-6b3e-46a0-a4d3-d6744640d144 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:57:34.416839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-24633741-6b3e-46a0-a4d3-d6744640d144 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:57:34.416839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:34.416839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:34.416839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:34.416839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:34.416839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:34.416839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:34.416839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:34.416839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:34.416839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:34.416839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:34.416839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:34.416839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:57:34.416839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:57:34.416839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:57:34.416839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:57:34.417705 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:57:34.417705 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:57:34.417705 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:57:34.417705 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:57:34.417705 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:34.417705 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:34.417705 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:34.417705 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:34.417705 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:34.417705 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:34.417705 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:34.417705 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:34.417705 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:34.417705 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:34.417705 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:34.417705 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:34.418554 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:34.418554 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:34.418554 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:34.418554 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:34.418554 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:57:34.418554 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:57:34.418554 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:57:34.418554 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:57:34.418554 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:57:34.418554 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:57:34.418554 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:57:34.418554 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:57:34.418554 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 35/69] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 15:57:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:57:34.419365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4409395-fc6d-462d-8ffa-13cec01c361c None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:34.419641 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4409395-fc6d-462d-8ffa-13cec01c361c None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:34.420037 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4409395-fc6d-462d-8ffa-13cec01c361c None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:34.421817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 36/70] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:34.422560 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af52165-79a1-478a-83e5-40a5536a8839 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:34.422855 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af52165-79a1-478a-83e5-40a5536a8839 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:34.423182 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af52165-79a1-478a-83e5-40a5536a8839 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:34.424437 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 37/71] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:34.426801 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af52165-79a1-478a-83e5-40a5536a8839 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:34.492660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96633419-76bb-4152-898b-fa8f635c628a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:34.526903 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a701844-ee49-443a-ad87-0050102d3b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:34.528307 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a701844-ee49-443a-ad87-0050102d3b7f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:34.528701 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a701844-ee49-443a-ad87-0050102d3b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:34.528992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a701844-ee49-443a-ad87-0050102d3b7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:34.538239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af52165-79a1-478a-83e5-40a5536a8839 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:34.539930 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a701844-ee49-443a-ad87-0050102d3b7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:57:34.547626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-2a701844-ee49-443a-ad87-0050102d3b7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:57:34.547626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:34.547626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:34.547626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:34.547626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:34.547626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:34.547626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:34.547626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:34.547626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:34.547626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:34.547626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:34.547626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:34.547626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:57:34.547626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:57:34.547626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:57:34.547626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:57:34.548642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:57:34.548642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:57:34.548642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:57:34.548642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:57:34.548642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:34.548642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:34.548642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:34.548642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:34.548642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:34.548642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:34.548642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:34.548642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:34.548642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:34.548642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:34.548642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:34.548642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:34.554515 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:34.554515 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:34.554515 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:34.554515 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:34.554515 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:57:34.554515 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:57:34.554515 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:57:34.554515 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:57:34.554515 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:57:34.554515 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:57:34.554515 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:57:34.554515 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:57:34.554515 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 35/72] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 15:57:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:57:34.554515 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a701844-ee49-443a-ad87-0050102d3b7f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:34.568365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:34.570506 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:34.574933 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a50de2-a520-412f-b479-b58adbe91341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:34.575597 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a50de2-a520-412f-b479-b58adbe91341 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:34.575887 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a50de2-a520-412f-b479-b58adbe91341 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:34.576166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a50de2-a520-412f-b479-b58adbe91341 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:34.576923 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a50de2-a520-412f-b479-b58adbe91341 None admin] RBAC: Adding query filter params (name=member) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:34.577230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a50de2-a520-412f-b479-b58adbe91341 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:34.578677 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a50de2-a520-412f-b479-b58adbe91341 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:34.588593 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 38/73] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 15:57:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:34.589788 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42215358-3e84-428a-9906-d185bb20a5d1 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:34.590105 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42215358-3e84-428a-9906-d185bb20a5d1 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:34.590391 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42215358-3e84-428a-9906-d185bb20a5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:34.613399 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-42215358-3e84-428a-9906-d185bb20a5d1 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:34.614640 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-42215358-3e84-428a-9906-d185bb20a5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:34.657955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a701844-ee49-443a-ad87-0050102d3b7f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:34.685285 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 39/74] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:34.686696 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe2980-10bd-4958-8270-49ae096b1840 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:34.686997 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe2980-10bd-4958-8270-49ae096b1840 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:34.687285 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe2980-10bd-4958-8270-49ae096b1840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:34.693800 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71672e76-706d-46f2-93e6-afb764832a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:34.694617 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71672e76-706d-46f2-93e6-afb764832a5f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:34.694906 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71672e76-706d-46f2-93e6-afb764832a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:34.695190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71672e76-706d-46f2-93e6-afb764832a5f None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:34.695954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71672e76-706d-46f2-93e6-afb764832a5f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:34.696265 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71672e76-706d-46f2-93e6-afb764832a5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:34.697664 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71672e76-706d-46f2-93e6-afb764832a5f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:34.711372 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 36/75] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:57:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:34.711372 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71672e76-706d-46f2-93e6-afb764832a5f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:34.715385 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-9cbe2980-10bd-4958-8270-49ae096b1840 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:34.716613 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-9cbe2980-10bd-4958-8270-49ae096b1840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:34.792668 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 40/76] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:34.793546 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe2980-10bd-4958-8270-49ae096b1840 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:34.818829 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:34.839955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71672e76-706d-46f2-93e6-afb764832a5f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:34.850820 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:34.875374 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db621da-00e3-47e2-b47b-f85a99853430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:34.876060 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db621da-00e3-47e2-b47b-f85a99853430 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:34.876346 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db621da-00e3-47e2-b47b-f85a99853430 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:34.876629 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db621da-00e3-47e2-b47b-f85a99853430 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:34.877379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db621da-00e3-47e2-b47b-f85a99853430 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:34.877690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db621da-00e3-47e2-b47b-f85a99853430 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:34.879124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db621da-00e3-47e2-b47b-f85a99853430 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:34.888488 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 37/77] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:57:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:34.889138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db621da-00e3-47e2-b47b-f85a99853430 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:34.904055 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe2980-10bd-4958-8270-49ae096b1840 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:34.940460 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88760a6-26d8-4079-9485-08e5f07857b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:34.941508 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88760a6-26d8-4079-9485-08e5f07857b2 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:34.941635 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88760a6-26d8-4079-9485-08e5f07857b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:34.941897 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88760a6-26d8-4079-9485-08e5f07857b2 None admin] PATH_INFO: `/v3/users/26f5b3d45f5048aa9a49fb86df108d90` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:34.956142 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88760a6-26d8-4079-9485-08e5f07857b2 None admin] RBAC: Authorizing `identity:get_user(user_id=26f5b3d45f5048aa9a49fb86df108d90)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:34.957633 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88760a6-26d8-4079-9485-08e5f07857b2 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:34.959432 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 41/78] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:57:34 2022] GET /identity/v3/users/26f5b3d45f5048aa9a49fb86df108d90 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:34.960201 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88760a6-26d8-4079-9485-08e5f07857b2 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:35.006433 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db621da-00e3-47e2-b47b-f85a99853430 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:35.041735 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76db856-44e7-42ab-944b-898868ea5209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:35.042774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76db856-44e7-42ab-944b-898868ea5209 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:35.043576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76db856-44e7-42ab-944b-898868ea5209 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:35.044068 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76db856-44e7-42ab-944b-898868ea5209 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:35.045688 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76db856-44e7-42ab-944b-898868ea5209 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:35.047091 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76db856-44e7-42ab-944b-898868ea5209 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:35.049291 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 38/79] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:57:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:35.049689 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76db856-44e7-42ab-944b-898868ea5209 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:35.065364 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:35.072622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88760a6-26d8-4079-9485-08e5f07857b2 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:35.084177 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:35.109286 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303379b6-2179-45c0-833a-6b025bfbbcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:35.110197 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303379b6-2179-45c0-833a-6b025bfbbcaa None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:35.110483 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303379b6-2179-45c0-833a-6b025bfbbcaa None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:35.110761 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303379b6-2179-45c0-833a-6b025bfbbcaa None admin] PATH_INFO: `/v3/users/26f5b3d45f5048aa9a49fb86df108d90` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:35.111970 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303379b6-2179-45c0-833a-6b025bfbbcaa None admin] RBAC: Authorizing `identity:get_user(user_id=26f5b3d45f5048aa9a49fb86df108d90)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:35.113431 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303379b6-2179-45c0-833a-6b025bfbbcaa None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:35.115106 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 42/80] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:57:34 2022] GET /identity/v3/users/26f5b3d45f5048aa9a49fb86df108d90 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:35.115870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303379b6-2179-45c0-833a-6b025bfbbcaa None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:35.162546 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76db856-44e7-42ab-944b-898868ea5209 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:35.198276 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f7fd1-2093-48c9-9f92-e060b48b9575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:35.199075 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f7fd1-2093-48c9-9f92-e060b48b9575 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:35.199370 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f7fd1-2093-48c9-9f92-e060b48b9575 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:35.199642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f7fd1-2093-48c9-9f92-e060b48b9575 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:35.209854 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f7fd1-2093-48c9-9f92-e060b48b9575 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:57:35.217700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-ca5f7fd1-2093-48c9-9f92-e060b48b9575 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:57:35.217700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:35.217700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:35.217700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:35.217700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:35.217700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:35.217700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:35.217700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:35.217700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:35.217700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:35.217700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:35.217700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:35.217700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:57:35.217700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:57:35.217700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:57:35.217700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:57:35.218565 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:57:35.218565 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:57:35.218565 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:57:35.218565 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:57:35.218565 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:35.218565 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:35.218565 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:35.218565 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:35.218565 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:35.218565 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:35.218565 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:35.218565 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:35.218565 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:35.218565 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:35.218565 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:35.218565 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:35.219663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:35.219663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:35.219663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:35.219663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:35.219663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:57:35.219663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:57:35.219663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:57:35.219663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:57:35.219663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:57:35.219663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:57:35.219663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:57:35.219663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:57:35.219663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 39/81] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:57:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:57:35.220485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f7fd1-2093-48c9-9f92-e060b48b9575 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:35.223114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303379b6-2179-45c0-833a-6b025bfbbcaa None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:35.260722 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063a249-8196-4f9a-8ffe-0906fff1f6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:35.262664 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063a249-8196-4f9a-8ffe-0906fff1f6cf None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:35.262959 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063a249-8196-4f9a-8ffe-0906fff1f6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:35.263237 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063a249-8196-4f9a-8ffe-0906fff1f6cf None admin] PATH_INFO: `/v3/users/9593d0f94eed4e3e83f1323aab2716a7` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:35.264957 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4063a249-8196-4f9a-8ffe-0906fff1f6cf None admin] RBAC: Authorizing `identity:get_user(user_id=9593d0f94eed4e3e83f1323aab2716a7)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:35.266002 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4063a249-8196-4f9a-8ffe-0906fff1f6cf None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:35.267679 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 43/82] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:57:35 2022] GET /identity/v3/users/9593d0f94eed4e3e83f1323aab2716a7 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:35.268428 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063a249-8196-4f9a-8ffe-0906fff1f6cf None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:35.290273 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:35.321215 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:35.325480 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f7fd1-2093-48c9-9f92-e060b48b9575 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:35.362095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64674b39-dafa-43c2-a73a-8ec7f5b65b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:35.362758 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64674b39-dafa-43c2-a73a-8ec7f5b65b9e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:35.363051 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64674b39-dafa-43c2-a73a-8ec7f5b65b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:35.363336 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64674b39-dafa-43c2-a73a-8ec7f5b65b9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:35.376342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063a249-8196-4f9a-8ffe-0906fff1f6cf None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:35.411104 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17adfa5-b795-44d0-ad2d-53fdcffb972b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:35.411776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17adfa5-b795-44d0-ad2d-53fdcffb972b None admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:35.412285 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17adfa5-b795-44d0-ad2d-53fdcffb972b None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:35.412389 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17adfa5-b795-44d0-ad2d-53fdcffb972b None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:35.413058 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17adfa5-b795-44d0-ad2d-53fdcffb972b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:35.414591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17adfa5-b795-44d0-ad2d-53fdcffb972b None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:35.433628 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64674b39-dafa-43c2-a73a-8ec7f5b65b9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:35.435066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 44/83] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 15:57:35 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:57:35.435650 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64674b39-dafa-43c2-a73a-8ec7f5b65b9e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:35.435899 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17adfa5-b795-44d0-ad2d-53fdcffb972b None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:35.518664 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:35.557274 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17adfa5-b795-44d0-ad2d-53fdcffb972b None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:35.562624 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 40/84] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:57:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:57:35.562624 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64674b39-dafa-43c2-a73a-8ec7f5b65b9e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:35.563804 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:35.606852 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166c9b2-0922-4ffd-9d30-fdb6d60e58ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:35.607523 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166c9b2-0922-4ffd-9d30-fdb6d60e58ff None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:35.608524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166c9b2-0922-4ffd-9d30-fdb6d60e58ff None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:35.608524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166c9b2-0922-4ffd-9d30-fdb6d60e58ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:35.682470 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64674b39-dafa-43c2-a73a-8ec7f5b65b9e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:35.703709 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166c9b2-0922-4ffd-9d30-fdb6d60e58ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:35.705229 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166c9b2-0922-4ffd-9d30-fdb6d60e58ff None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:35.732021 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57688909-2222-42aa-bd40-4872d97bb00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:35.733068 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57688909-2222-42aa-bd40-4872d97bb00f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:35.734088 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57688909-2222-42aa-bd40-4872d97bb00f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:35.734582 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57688909-2222-42aa-bd40-4872d97bb00f None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:35.735555 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57688909-2222-42aa-bd40-4872d97bb00f None admin] RBAC: Adding query filter params (name=member) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:35.736244 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57688909-2222-42aa-bd40-4872d97bb00f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:35.738202 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57688909-2222-42aa-bd40-4872d97bb00f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:35.749549 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 41/85] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 15:57:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:35.750288 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57688909-2222-42aa-bd40-4872d97bb00f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:35.793902 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:35.830016 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 45/86] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:57:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:57:35.834819 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166c9b2-0922-4ffd-9d30-fdb6d60e58ff None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:35.837378 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:35.886698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57688909-2222-42aa-bd40-4872d97bb00f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:35.925056 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a391e7c-1187-4785-97c0-89b377758d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:35.926458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a391e7c-1187-4785-97c0-89b377758d59 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:35.926888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a391e7c-1187-4785-97c0-89b377758d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:35.927325 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a391e7c-1187-4785-97c0-89b377758d59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:35.944388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166c9b2-0922-4ffd-9d30-fdb6d60e58ff None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:35.981863 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6571b3-d4ef-433b-ab87-be567df08149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:35.982949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6571b3-d4ef-433b-ab87-be567df08149 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:35.983340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6571b3-d4ef-433b-ab87-be567df08149 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:35.983722 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6571b3-d4ef-433b-ab87-be567df08149 None admin] PATH_INFO: `/v3/projects/a83dc386fda8411eb2d4aef8e43b7065` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:35.985164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6571b3-d4ef-433b-ab87-be567df08149 None admin] RBAC: Authorizing `identity:get_project(project_id=a83dc386fda8411eb2d4aef8e43b7065)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:35.986912 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6571b3-d4ef-433b-ab87-be567df08149 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:35.994379 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 46/87] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:57:35 2022] GET /identity/v3/projects/a83dc386fda8411eb2d4aef8e43b7065 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:35.994379 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6571b3-d4ef-433b-ab87-be567df08149 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:36.014047 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a391e7c-1187-4785-97c0-89b377758d59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:36.015922 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a391e7c-1187-4785-97c0-89b377758d59 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:36.050301 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:36.068662 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:36.111356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6571b3-d4ef-433b-ab87-be567df08149 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:36.141976 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 42/88] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:57:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:57:36.143119 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a391e7c-1187-4785-97c0-89b377758d59 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:36.154659 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875ea92-566b-462f-bea5-574339df14c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:36.155927 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875ea92-566b-462f-bea5-574339df14c4 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:36.156452 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875ea92-566b-462f-bea5-574339df14c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:36.156926 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875ea92-566b-462f-bea5-574339df14c4 None admin] PATH_INFO: `/v3/users/26f5b3d45f5048aa9a49fb86df108d90` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:36.158414 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875ea92-566b-462f-bea5-574339df14c4 None admin] RBAC: Authorizing `identity:get_user(user_id=26f5b3d45f5048aa9a49fb86df108d90)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:36.160079 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875ea92-566b-462f-bea5-574339df14c4 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:36.161965 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 47/89] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:57:35 2022] GET /identity/v3/users/26f5b3d45f5048aa9a49fb86df108d90 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:36.162935 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875ea92-566b-462f-bea5-574339df14c4 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:36.273695 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875ea92-566b-462f-bea5-574339df14c4 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:36.278682 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a391e7c-1187-4785-97c0-89b377758d59 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:36.314682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7a695-b4f0-493f-a5bd-cad96856a7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:36.315413 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7a695-b4f0-493f-a5bd-cad96856a7e6 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:36.315700 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7a695-b4f0-493f-a5bd-cad96856a7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:36.315980 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7a695-b4f0-493f-a5bd-cad96856a7e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:36.316873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7a695-b4f0-493f-a5bd-cad96856a7e6 None admin] RBAC: Adding query filter params (role.id=3ce570490865493380e6941d004136bd, scope.project.id=a83dc386fda8411eb2d4aef8e43b7065, user.id=26f5b3d45f5048aa9a49fb86df108d90) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:36.317193 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7a695-b4f0-493f-a5bd-cad96856a7e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:36.318696 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7a695-b4f0-493f-a5bd-cad96856a7e6 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:36.319004 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:36.325558 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 48/90] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:57:36 2022] GET /identity/v3/role_assignments?user.id=26f5b3d45f5048aa9a49fb86df108d90&scope.project.id=a83dc386fda8411eb2d4aef8e43b7065&role.id=3ce570490865493380e6941d004136bd => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:36.326426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7a695-b4f0-493f-a5bd-cad96856a7e6 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:36.326532 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a16265-affa-41a0-acdb-705b887f2df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:36.327599 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a16265-affa-41a0-acdb-705b887f2df4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:36.327892 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a16265-affa-41a0-acdb-705b887f2df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:36.328179 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a16265-affa-41a0-acdb-705b887f2df4 None admin] PATH_INFO: `/v3/projects/a83dc386fda8411eb2d4aef8e43b7065` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:36.329207 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:36.329427 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a16265-affa-41a0-acdb-705b887f2df4 None admin] RBAC: Authorizing `identity:get_project(project_id=a83dc386fda8411eb2d4aef8e43b7065)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:36.331192 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a16265-affa-41a0-acdb-705b887f2df4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:36.338028 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 43/91] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:57:36 2022] GET /identity/v3/projects/a83dc386fda8411eb2d4aef8e43b7065 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:36.339083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a16265-affa-41a0-acdb-705b887f2df4 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:36.447557 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7a695-b4f0-493f-a5bd-cad96856a7e6 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:36.485052 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a16265-affa-41a0-acdb-705b887f2df4 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:36.490382 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a4727-ae9a-45b0-85c1-3c4fdee08cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:36.491266 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a4727-ae9a-45b0-85c1-3c4fdee08cfb None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:36.491554 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a4727-ae9a-45b0-85c1-3c4fdee08cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:36.491838 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a4727-ae9a-45b0-85c1-3c4fdee08cfb None admin] PATH_INFO: `/v3/projects/a83dc386fda8411eb2d4aef8e43b7065` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:36.493102 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a4727-ae9a-45b0-85c1-3c4fdee08cfb None admin] RBAC: Authorizing `identity:get_project(project_id=a83dc386fda8411eb2d4aef8e43b7065)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:36.495354 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a4727-ae9a-45b0-85c1-3c4fdee08cfb None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:36.498361 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 49/92] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:57:36 2022] GET /identity/v3/projects/a83dc386fda8411eb2d4aef8e43b7065 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:36.500139 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a4727-ae9a-45b0-85c1-3c4fdee08cfb None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:36.528181 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94728993-09b9-4376-9f02-e48d3b7182be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:36.529051 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94728993-09b9-4376-9f02-e48d3b7182be None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:36.529407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94728993-09b9-4376-9f02-e48d3b7182be None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:36.529798 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94728993-09b9-4376-9f02-e48d3b7182be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:36.597887 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:36.618545 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:36.622672 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94728993-09b9-4376-9f02-e48d3b7182be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:36.625135 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94728993-09b9-4376-9f02-e48d3b7182be None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:36.661937 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a4727-ae9a-45b0-85c1-3c4fdee08cfb None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:36.705394 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e31279d-0100-460d-a540-d0689a2e509e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:36.706468 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31279d-0100-460d-a540-d0689a2e509e None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:36.706960 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31279d-0100-460d-a540-d0689a2e509e None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:36.707441 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31279d-0100-460d-a540-d0689a2e509e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:36.709217 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e31279d-0100-460d-a540-d0689a2e509e None admin] RBAC: Adding query filter params (role.id=e0b0c216ef51441ab34b6e036da09daa, scope.project.id=a83dc386fda8411eb2d4aef8e43b7065, user.id=26f5b3d45f5048aa9a49fb86df108d90) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:36.709789 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e31279d-0100-460d-a540-d0689a2e509e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:36.712025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e31279d-0100-460d-a540-d0689a2e509e None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:36.719595 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 50/93] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:57:36 2022] GET /identity/v3/role_assignments?user.id=26f5b3d45f5048aa9a49fb86df108d90&scope.project.id=a83dc386fda8411eb2d4aef8e43b7065&role.id=e0b0c216ef51441ab34b6e036da09daa => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:36.720826 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e31279d-0100-460d-a540-d0689a2e509e None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:36.840820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 44/94] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:57:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:57:36.846344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94728993-09b9-4376-9f02-e48d3b7182be None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:36.863280 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e31279d-0100-460d-a540-d0689a2e509e None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:36.904084 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e2e35-c9ed-4260-845a-6d61c2495b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:36.905257 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e2e35-c9ed-4260-845a-6d61c2495b21 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:36.905785 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e2e35-c9ed-4260-845a-6d61c2495b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:36.906379 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e2e35-c9ed-4260-845a-6d61c2495b21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:36.907431 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879e2e35-c9ed-4260-845a-6d61c2495b21 None admin] RBAC: Adding query filter params (role.id=bbbc77ec645d4ff5a8bf29ce0377f578, scope.project.id=a83dc386fda8411eb2d4aef8e43b7065, user.id=9593d0f94eed4e3e83f1323aab2716a7) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:36.907949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879e2e35-c9ed-4260-845a-6d61c2495b21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:36.909622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879e2e35-c9ed-4260-845a-6d61c2495b21 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:36.911989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:36.917757 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 51/95] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:57:36 2022] GET /identity/v3/role_assignments?user.id=9593d0f94eed4e3e83f1323aab2716a7&scope.project.id=a83dc386fda8411eb2d4aef8e43b7065&role.id=bbbc77ec645d4ff5a8bf29ce0377f578 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:37.012008 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:37.018122 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94728993-09b9-4376-9f02-e48d3b7182be None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:37.071824 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0febefe-90dc-4ee5-87ea-b1858ec2a2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:37.073090 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0febefe-90dc-4ee5-87ea-b1858ec2a2d5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:37.073578 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0febefe-90dc-4ee5-87ea-b1858ec2a2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:37.074094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0febefe-90dc-4ee5-87ea-b1858ec2a2d5 None admin] PATH_INFO: `/v3/projects/c2db1b3048a147849ab9d1d46779d24b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:37.078780 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0febefe-90dc-4ee5-87ea-b1858ec2a2d5 None admin] RBAC: Authorizing `identity:get_project(project_id=c2db1b3048a147849ab9d1d46779d24b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:37.080558 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0febefe-90dc-4ee5-87ea-b1858ec2a2d5 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:37.089770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 45/96] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:57:36 2022] GET /identity/v3/projects/c2db1b3048a147849ab9d1d46779d24b => generated 312 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:37.096729 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e2e35-c9ed-4260-845a-6d61c2495b21 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:37.214792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e2e35-c9ed-4260-845a-6d61c2495b21 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:37.253732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ba955-2248-40e5-91b6-c8740eb24278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:37.255101 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ba955-2248-40e5-91b6-c8740eb24278 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:37.255605 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ba955-2248-40e5-91b6-c8740eb24278 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:37.256086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ba955-2248-40e5-91b6-c8740eb24278 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:37.257133 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ba955-2248-40e5-91b6-c8740eb24278 None admin] RBAC: Adding query filter params (role.id=3ce570490865493380e6941d004136bd, scope.project.id=c2db1b3048a147849ab9d1d46779d24b, user.id=26f5b3d45f5048aa9a49fb86df108d90) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:37.257657 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ba955-2248-40e5-91b6-c8740eb24278 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:37.259380 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ba955-2248-40e5-91b6-c8740eb24278 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:37.266536 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 52/97] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:57:37 2022] GET /identity/v3/role_assignments?user.id=26f5b3d45f5048aa9a49fb86df108d90&scope.project.id=c2db1b3048a147849ab9d1d46779d24b&role.id=3ce570490865493380e6941d004136bd => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:38.051418 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6ea1c-a403-480a-bd3d-e95df19116a2 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:38.051922 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6ea1c-a403-480a-bd3d-e95df19116a2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:38.052205 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6ea1c-a403-480a-bd3d-e95df19116a2 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:38.053676 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 46/98] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:38.060813 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe390e1-fe86-4543-91ed-55fa16565fe8 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:38.061269 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe390e1-fe86-4543-91ed-55fa16565fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:38.061727 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe390e1-fe86-4543-91ed-55fa16565fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:38.086104 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-cbe390e1-fe86-4543-91ed-55fa16565fe8 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:38.087326 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-cbe390e1-fe86-4543-91ed-55fa16565fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:38.174986 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:38.192565 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 53/99] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:38.227498 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6ea1c-a403-480a-bd3d-e95df19116a2 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:38.350938 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6ea1c-a403-480a-bd3d-e95df19116a2 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:38.387598 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4dbe5-1dd6-4228-a635-5897457821ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:38.388481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4dbe5-1dd6-4228-a635-5897457821ca None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:38.388769 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4dbe5-1dd6-4228-a635-5897457821ca None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:38.389104 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4dbe5-1dd6-4228-a635-5897457821ca None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:38.400266 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4dbe5-1dd6-4228-a635-5897457821ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:57:38.408919 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-d9b4dbe5-1dd6-4228-a635-5897457821ca None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:57:38.408919 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:38.408919 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:38.408919 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:38.408919 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:38.408919 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:38.408919 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:38.408919 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:38.408919 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:38.408919 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:38.408919 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:38.408919 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:38.408919 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:57:38.408919 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:57:38.408919 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:57:38.408919 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:57:38.409795 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:57:38.409795 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:57:38.409795 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:57:38.409795 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:57:38.409795 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:38.409795 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:38.409795 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:38.409795 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:38.409795 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:38.409795 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:38.409795 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:38.409795 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:38.409795 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:38.409795 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:38.409795 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:38.409795 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:38.410657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:38.410657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:38.410657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:38.410657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:38.410657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:57:38.410657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:57:38.410657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:57:38.410657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:57:38.410657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:57:38.410657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:57:38.410657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:57:38.410657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:57:38.410657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 47/100] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:57:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:57:38.416180 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe390e1-fe86-4543-91ed-55fa16565fe8 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:38.554238 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe390e1-fe86-4543-91ed-55fa16565fe8 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:38.601900 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea5524-72cd-43e5-b39e-0f1baa23b0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:38.602982 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea5524-72cd-43e5-b39e-0f1baa23b0e8 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:38.603482 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea5524-72cd-43e5-b39e-0f1baa23b0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:38.603969 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea5524-72cd-43e5-b39e-0f1baa23b0e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:38.604974 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea5524-72cd-43e5-b39e-0f1baa23b0e8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:38.605506 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea5524-72cd-43e5-b39e-0f1baa23b0e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:38.607960 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea5524-72cd-43e5-b39e-0f1baa23b0e8 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:38.617521 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 54/101] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 15:57:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:38.626057 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4dbe5-1dd6-4228-a635-5897457821ca None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:38.667381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:38.751219 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4dbe5-1dd6-4228-a635-5897457821ca None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:38.796920 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a98d3-0eac-4213-a96b-fbc97105c895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:38.798065 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a98d3-0eac-4213-a96b-fbc97105c895 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:38.798514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a98d3-0eac-4213-a96b-fbc97105c895 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:38.798950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a98d3-0eac-4213-a96b-fbc97105c895 None admin] PATH_INFO: `/v3/users/059108b974dc423a92557b010d5fbde9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:38.816984 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a98d3-0eac-4213-a96b-fbc97105c895 None admin] RBAC: Authorizing `identity:get_user(user_id=059108b974dc423a92557b010d5fbde9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:38.818834 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a98d3-0eac-4213-a96b-fbc97105c895 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:38.820830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 48/102] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:57:38 2022] GET /identity/v3/users/059108b974dc423a92557b010d5fbde9 => generated 289 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:38.827543 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea5524-72cd-43e5-b39e-0f1baa23b0e8 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:38.920276 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:38.985691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea5524-72cd-43e5-b39e-0f1baa23b0e8 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:39.025254 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce8c13-af85-490e-a9aa-0da2356d82cf None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:39.025889 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce8c13-af85-490e-a9aa-0da2356d82cf None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:39.026427 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce8c13-af85-490e-a9aa-0da2356d82cf None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:39.028324 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 49/103] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:39.032065 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cdf5a7-fe4a-4215-9409-c3e4f10b1eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:39.032728 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cdf5a7-fe4a-4215-9409-c3e4f10b1eff None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:39.033015 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cdf5a7-fe4a-4215-9409-c3e4f10b1eff None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:39.033302 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cdf5a7-fe4a-4215-9409-c3e4f10b1eff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:39.034772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9164a-0261-42e1-828d-d6eda8c3d530 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:39.035267 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9164a-0261-42e1-828d-d6eda8c3d530 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:39.035752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9164a-0261-42e1-828d-d6eda8c3d530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:39.071070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-98e9164a-0261-42e1-828d-d6eda8c3d530 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:39.073312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-98e9164a-0261-42e1-828d-d6eda8c3d530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:39.175212 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cdf5a7-fe4a-4215-9409-c3e4f10b1eff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:39.177392 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cdf5a7-fe4a-4215-9409-c3e4f10b1eff None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:39.228608 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 50/104] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:57:39.276221 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9164a-0261-42e1-828d-d6eda8c3d530 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:39.287511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:39.310989 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:39.349473 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 55/105] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:57:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:57:39.357083 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cdf5a7-fe4a-4215-9409-c3e4f10b1eff None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:39.488305 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cdf5a7-fe4a-4215-9409-c3e4f10b1eff None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:39.522935 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9164a-0261-42e1-828d-d6eda8c3d530 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:39.573824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f307c23a-46f0-4a4b-bf65-6190c7f5332d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:39.574759 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307c23a-46f0-4a4b-bf65-6190c7f5332d None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:39.575053 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307c23a-46f0-4a4b-bf65-6190c7f5332d None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:39.575337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307c23a-46f0-4a4b-bf65-6190c7f5332d None admin] PATH_INFO: `/v3/projects/a83dc386fda8411eb2d4aef8e43b7065` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:39.577316 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f307c23a-46f0-4a4b-bf65-6190c7f5332d None admin] RBAC: Authorizing `identity:get_project(project_id=a83dc386fda8411eb2d4aef8e43b7065)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:39.578821 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f307c23a-46f0-4a4b-bf65-6190c7f5332d None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:39.581648 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 56/106] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:57:39 2022] GET /identity/v3/projects/a83dc386fda8411eb2d4aef8e43b7065 => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:39.587056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbe82c-f0d4-4ad8-a488-ca3785c3fa05 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:39.587348 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbe82c-f0d4-4ad8-a488-ca3785c3fa05 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:39.587637 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbe82c-f0d4-4ad8-a488-ca3785c3fa05 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:39.591257 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 57/107] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:39.591866 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbe82c-f0d4-4ad8-a488-ca3785c3fa05 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:39.597044 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54a9494-2492-43ca-9668-4d226b367f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:39.597865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a9494-2492-43ca-9668-4d226b367f58 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:39.598185 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a9494-2492-43ca-9668-4d226b367f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:39.598468 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a9494-2492-43ca-9668-4d226b367f58 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:39.599784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54a9494-2492-43ca-9668-4d226b367f58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:39.601259 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54a9494-2492-43ca-9668-4d226b367f58 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:39.611083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 51/108] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:57:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:39.612476 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b374cb-4d2d-41fa-96ae-5c5f132762aa None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:39.612771 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b374cb-4d2d-41fa-96ae-5c5f132762aa None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:39.613058 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b374cb-4d2d-41fa-96ae-5c5f132762aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:39.646420 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:39.648823 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-57b374cb-4d2d-41fa-96ae-5c5f132762aa None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:39.650305 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-57b374cb-4d2d-41fa-96ae-5c5f132762aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:39.806139 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 52/109] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:39.806992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b374cb-4d2d-41fa-96ae-5c5f132762aa None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:39.817620 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbe82c-f0d4-4ad8-a488-ca3785c3fa05 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:39.843489 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:39.906747 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7060c-4dcb-48cb-92fa-6f3da4bb7370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:39.907853 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7060c-4dcb-48cb-92fa-6f3da4bb7370 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:39.909891 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7060c-4dcb-48cb-92fa-6f3da4bb7370 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:39.910396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7060c-4dcb-48cb-92fa-6f3da4bb7370 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:39.911773 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7060c-4dcb-48cb-92fa-6f3da4bb7370 None admin] RBAC: Adding query filter params (role.id=1a524a543caa49b286168391bb19d71e, scope.project.id=a83dc386fda8411eb2d4aef8e43b7065, user.id=059108b974dc423a92557b010d5fbde9) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:39.913094 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7060c-4dcb-48cb-92fa-6f3da4bb7370 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:39.915020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7060c-4dcb-48cb-92fa-6f3da4bb7370 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:39.933006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 58/110] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:57:39 2022] GET /identity/v3/role_assignments?user.id=059108b974dc423a92557b010d5fbde9&scope.project.id=a83dc386fda8411eb2d4aef8e43b7065&role.id=1a524a543caa49b286168391bb19d71e => generated 259 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:39.934508 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7060c-4dcb-48cb-92fa-6f3da4bb7370 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:40.010710 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b374cb-4d2d-41fa-96ae-5c5f132762aa None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:40.052647 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b18a2f-a11d-487c-b9b3-108787f91a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:40.053561 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b18a2f-a11d-487c-b9b3-108787f91a60 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:40.054066 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b18a2f-a11d-487c-b9b3-108787f91a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:40.057759 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b18a2f-a11d-487c-b9b3-108787f91a60 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:40.060208 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b18a2f-a11d-487c-b9b3-108787f91a60 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:40.062157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b18a2f-a11d-487c-b9b3-108787f91a60 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:40.114285 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 53/111] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jun 16 15:57:39 2022] POST /identity/v3/projects => generated 302 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:57:40.116465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc25942-f428-4fcf-8f7f-53b01e15553f None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:40.116948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc25942-f428-4fcf-8f7f-53b01e15553f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:40.117395 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc25942-f428-4fcf-8f7f-53b01e15553f None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:40.119322 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 54/112] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:40.124911 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de355dbd-ef55-4e40-8525-2969342cd7db None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:40.125446 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de355dbd-ef55-4e40-8525-2969342cd7db None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:40.125907 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de355dbd-ef55-4e40-8525-2969342cd7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:40.138621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:40.154782 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-de355dbd-ef55-4e40-8525-2969342cd7db None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:40.156052 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-de355dbd-ef55-4e40-8525-2969342cd7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:40.163625 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7060c-4dcb-48cb-92fa-6f3da4bb7370 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:40.282704 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed43e8-79a4-4f9f-88e1-72887ed1f72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:40.283574 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed43e8-79a4-4f9f-88e1-72887ed1f72a None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:40.283867 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed43e8-79a4-4f9f-88e1-72887ed1f72a None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:40.284151 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed43e8-79a4-4f9f-88e1-72887ed1f72a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:40.342149 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed43e8-79a4-4f9f-88e1-72887ed1f72a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:57:40.347584 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:40.358077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-d3ed43e8-79a4-4f9f-88e1-72887ed1f72a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:57:40.358077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:40.358077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:40.358077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:40.358077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:40.358077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:40.358077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:40.358077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:40.358077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:40.358077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:40.358077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:40.358077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:40.358077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:57:40.358077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:57:40.358077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:57:40.358077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:57:40.359325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:57:40.359325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:57:40.359325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:57:40.359325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:57:40.359325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:40.359325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:40.359325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:40.359325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:40.359325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:40.359325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:40.359325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:40.359325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:40.359325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:40.359325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:40.359325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:40.359325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:40.360091 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:40.360091 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:40.360091 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:40.360091 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:40.360091 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:57:40.360091 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:57:40.360091 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:57:40.360091 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:57:40.360091 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:57:40.360091 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:57:40.360091 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:57:40.360091 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:57:40.360091 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 59/113] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:57:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 425 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:57:40.371032 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed43e8-79a4-4f9f-88e1-72887ed1f72a None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:40.395540 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 55/114] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:40.438473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de355dbd-ef55-4e40-8525-2969342cd7db None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:40.552843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed43e8-79a4-4f9f-88e1-72887ed1f72a None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:40.592673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e118ce-7227-4e2b-9957-fcbd880dc03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:40.593267 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e118ce-7227-4e2b-9957-fcbd880dc03a None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:40.593554 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e118ce-7227-4e2b-9957-fcbd880dc03a None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:40.593833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e118ce-7227-4e2b-9957-fcbd880dc03a None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:40.594625 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e118ce-7227-4e2b-9957-fcbd880dc03a None admin] RBAC: Adding query filter params (name=member) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:40.594936 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e118ce-7227-4e2b-9957-fcbd880dc03a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:40.596768 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e118ce-7227-4e2b-9957-fcbd880dc03a None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:40.624604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 60/115] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 15:57:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:40.629380 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c90c77-2c81-4361-b164-83a7f74af6a2 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:40.629380 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c90c77-2c81-4361-b164-83a7f74af6a2 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:40.629380 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c90c77-2c81-4361-b164-83a7f74af6a2 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:40.631213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 61/116] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:40.632190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c90c77-2c81-4361-b164-83a7f74af6a2 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:40.653912 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:40.681465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de355dbd-ef55-4e40-8525-2969342cd7db None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:40.743334 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4e6dc-8961-472e-b28e-a1b98f17f88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:40.744245 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4e6dc-8961-472e-b28e-a1b98f17f88b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:40.744540 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4e6dc-8961-472e-b28e-a1b98f17f88b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:40.745107 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4e6dc-8961-472e-b28e-a1b98f17f88b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:40.757420 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4e6dc-8961-472e-b28e-a1b98f17f88b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:57:40.765629 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-19b4e6dc-8961-472e-b28e-a1b98f17f88b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:57:40.765629 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:40.765629 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:40.765629 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:40.765629 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:40.765629 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:40.765629 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:40.765629 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:40.765629 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:40.765629 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:40.765629 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:40.765629 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:40.765629 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:57:40.765629 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:57:40.765629 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:57:40.765629 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:57:40.766567 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:57:40.766567 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:57:40.766567 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:57:40.766567 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:57:40.766567 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:40.766567 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:40.766567 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:40.766567 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:40.766567 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:40.766567 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:40.766567 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:40.766567 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:40.766567 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:40.766567 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:40.766567 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:40.766567 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:40.767349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:40.767349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:40.767349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:40.767349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:40.767349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:57:40.767349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:57:40.767349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:57:40.767349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:57:40.767349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:57:40.767349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:57:40.767349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:57:40.767349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:57:40.767349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 56/117] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 15:57:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:57:40.773012 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e2714-a023-477e-8d00-fd14a9e6b0f9 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:40.773012 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e2714-a023-477e-8d00-fd14a9e6b0f9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:40.773012 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e2714-a023-477e-8d00-fd14a9e6b0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:40.787605 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-265e2714-a023-477e-8d00-fd14a9e6b0f9 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:40.788883 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-265e2714-a023-477e-8d00-fd14a9e6b0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:40.800642 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c90c77-2c81-4361-b164-83a7f74af6a2 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:40.837722 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d81df-dcac-4684-baab-4fa0563162ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:40.838915 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d81df-dcac-4684-baab-4fa0563162ce None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:40.839379 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d81df-dcac-4684-baab-4fa0563162ce None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:40.839858 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d81df-dcac-4684-baab-4fa0563162ce None admin] PATH_INFO: `/v3/users/26f5b3d45f5048aa9a49fb86df108d90` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:40.841894 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d81df-dcac-4684-baab-4fa0563162ce None admin] RBAC: Authorizing `identity:get_user(user_id=26f5b3d45f5048aa9a49fb86df108d90)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:40.843653 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d81df-dcac-4684-baab-4fa0563162ce None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:40.845503 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 62/118] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:57:40 2022] GET /identity/v3/users/26f5b3d45f5048aa9a49fb86df108d90 => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:40.847074 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba8e5c-7ec4-48a3-951b-94f27ca77b25 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:40.847588 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba8e5c-7ec4-48a3-951b-94f27ca77b25 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:40.848081 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba8e5c-7ec4-48a3-951b-94f27ca77b25 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:40.849539 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 63/119] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:40.850299 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:40.850847 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba8e5c-7ec4-48a3-951b-94f27ca77b25 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:40.868551 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 57/120] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:40.869622 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e2714-a023-477e-8d00-fd14a9e6b0f9 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:40.944715 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:40.971657 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba8e5c-7ec4-48a3-951b-94f27ca77b25 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:40.986852 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e2714-a023-477e-8d00-fd14a9e6b0f9 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:41.011165 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b93490-fc82-4626-98d6-0ba797a62679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:41.012170 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b93490-fc82-4626-98d6-0ba797a62679 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:41.012631 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b93490-fc82-4626-98d6-0ba797a62679 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:41.013059 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b93490-fc82-4626-98d6-0ba797a62679 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:41.013998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b93490-fc82-4626-98d6-0ba797a62679 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:41.014450 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b93490-fc82-4626-98d6-0ba797a62679 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:41.016000 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b93490-fc82-4626-98d6-0ba797a62679 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:41.025170 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 64/121] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:57:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:41.026750 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068cf9d-bc12-4b12-9bc3-cb80c9144d90 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:41.027168 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068cf9d-bc12-4b12-9bc3-cb80c9144d90 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:41.027573 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068cf9d-bc12-4b12-9bc3-cb80c9144d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:41.029752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd21ee-f46e-4f2e-8690-7a13b3665e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:41.030962 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd21ee-f46e-4f2e-8690-7a13b3665e6f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:41.031265 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd21ee-f46e-4f2e-8690-7a13b3665e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:41.031555 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd21ee-f46e-4f2e-8690-7a13b3665e6f None admin] PATH_INFO: `/v3/projects/a83dc386fda8411eb2d4aef8e43b7065` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:41.033041 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd21ee-f46e-4f2e-8690-7a13b3665e6f None admin] RBAC: Authorizing `identity:get_project(project_id=a83dc386fda8411eb2d4aef8e43b7065)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:41.035063 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd21ee-f46e-4f2e-8690-7a13b3665e6f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:41.037440 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 58/122] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:57:40 2022] GET /identity/v3/projects/a83dc386fda8411eb2d4aef8e43b7065 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:41.038437 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd21ee-f46e-4f2e-8690-7a13b3665e6f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:41.048616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-3068cf9d-bc12-4b12-9bc3-cb80c9144d90 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:41.050106 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-3068cf9d-bc12-4b12-9bc3-cb80c9144d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:41.111860 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:41.136035 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 65/123] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:41.142094 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3068cf9d-bc12-4b12-9bc3-cb80c9144d90 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:41.182696 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd21ee-f46e-4f2e-8690-7a13b3665e6f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:41.220911 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ec7a8-c766-4588-845c-402854c94c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:41.221982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ec7a8-c766-4588-845c-402854c94c28 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:41.222229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ec7a8-c766-4588-845c-402854c94c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:41.222494 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ec7a8-c766-4588-845c-402854c94c28 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:41.236130 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843ec7a8-c766-4588-845c-402854c94c28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:57:41.244001 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:41.245085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-843ec7a8-c766-4588-845c-402854c94c28 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:57:41.245085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:41.245085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:41.245085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:41.245085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:41.245085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:41.245085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:41.245085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:41.245085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:41.245085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:41.245085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:41.245085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:41.245085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:57:41.245085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:57:41.245085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:57:41.245085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:57:41.246312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:57:41.246312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:57:41.246312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:57:41.246312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:57:41.246312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:41.246312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:41.246312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:41.246312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:41.246312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:41.246312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:41.246312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:41.246312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:41.246312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:41.246312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:41.246312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:41.246312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:41.248405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:41.248405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:41.248405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:41.248405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:41.248405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:57:41.248405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:57:41.248405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:57:41.248405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:57:41.248405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:57:41.248405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:57:41.248405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:57:41.248405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:57:41.250393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 59/124] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:57:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:57:41.252140 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ec7a8-c766-4588-845c-402854c94c28 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:41.290427 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3068cf9d-bc12-4b12-9bc3-cb80c9144d90 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:41.336632 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4097836c-cc38-433b-a5b9-988f9b94df9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:41.337793 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097836c-cc38-433b-a5b9-988f9b94df9b None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:41.338134 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097836c-cc38-433b-a5b9-988f9b94df9b None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:41.338413 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097836c-cc38-433b-a5b9-988f9b94df9b None admin] PATH_INFO: `/v3/users/26f5b3d45f5048aa9a49fb86df108d90` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:41.341084 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4097836c-cc38-433b-a5b9-988f9b94df9b None admin] RBAC: Authorizing `identity:get_user(user_id=26f5b3d45f5048aa9a49fb86df108d90)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:41.343201 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4097836c-cc38-433b-a5b9-988f9b94df9b None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:41.345785 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 66/125] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:57:41 2022] GET /identity/v3/users/26f5b3d45f5048aa9a49fb86df108d90 => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:41.346444 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4097836c-cc38-433b-a5b9-988f9b94df9b None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:41.369586 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ec7a8-c766-4588-845c-402854c94c28 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:41.388183 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:41.418142 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17182882-4def-4f38-b74d-17260c2344dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:41.419557 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17182882-4def-4f38-b74d-17260c2344dc None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:41.422214 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17182882-4def-4f38-b74d-17260c2344dc None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:41.422214 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17182882-4def-4f38-b74d-17260c2344dc None admin] PATH_INFO: `/v3/projects/a83dc386fda8411eb2d4aef8e43b7065/users/26f5b3d45f5048aa9a49fb86df108d90/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:41.423354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17182882-4def-4f38-b74d-17260c2344dc None admin] RBAC: Authorizing `identity:create_grant(project_id=a83dc386fda8411eb2d4aef8e43b7065, user_id=26f5b3d45f5048aa9a49fb86df108d90, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:41.425465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17182882-4def-4f38-b74d-17260c2344dc None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:41.441428 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 60/126] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 15:57:41 2022] PUT /identity/v3/projects/a83dc386fda8411eb2d4aef8e43b7065/users/26f5b3d45f5048aa9a49fb86df108d90/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:57:41.442575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17182882-4def-4f38-b74d-17260c2344dc None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:41.538265 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4097836c-cc38-433b-a5b9-988f9b94df9b None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:41.583066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b61991-92e4-4ab5-ac4e-71120e32a97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:41.583887 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b61991-92e4-4ab5-ac4e-71120e32a97e None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:41.584222 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b61991-92e4-4ab5-ac4e-71120e32a97e None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:41.584505 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b61991-92e4-4ab5-ac4e-71120e32a97e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:41.588304 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:41.597888 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b61991-92e4-4ab5-ac4e-71120e32a97e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:57:41.612424 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-60b61991-92e4-4ab5-ac4e-71120e32a97e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:57:41.612424 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:41.612424 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:41.612424 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:41.612424 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:41.612424 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:41.612424 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:41.612424 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:41.612424 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:41.612424 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:41.612424 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:41.612424 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:41.612424 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:57:41.612424 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:57:41.612424 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:57:41.612424 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:57:41.613551 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:57:41.613551 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:57:41.613551 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:57:41.613551 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:57:41.613551 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:41.613551 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:41.613551 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:41.613551 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:41.613551 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:41.613551 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:41.613551 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:41.613551 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:41.613551 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:41.613551 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:41.613551 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:41.613551 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:41.614542 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:41.614542 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:41.614542 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:41.614542 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:41.614542 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:57:41.614542 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:57:41.614542 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:57:41.614542 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:57:41.614542 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:57:41.614542 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:57:41.614542 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:57:41.614542 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:57:41.618329 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 67/127] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 15:57:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:57:41.619369 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b61991-92e4-4ab5-ac4e-71120e32a97e None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:41.620704 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17182882-4def-4f38-b74d-17260c2344dc None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:41.667967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1c424-2a08-42b7-a820-db15d6db45eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:41.669550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1c424-2a08-42b7-a820-db15d6db45eb None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:41.669898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1c424-2a08-42b7-a820-db15d6db45eb None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:41.671202 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1c424-2a08-42b7-a820-db15d6db45eb None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:41.672159 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1c424-2a08-42b7-a820-db15d6db45eb None admin] RBAC: Adding query filter params (name=member) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:41.672476 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1c424-2a08-42b7-a820-db15d6db45eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:41.673881 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1c424-2a08-42b7-a820-db15d6db45eb None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:41.684319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 61/128] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 15:57:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:41.685062 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1c424-2a08-42b7-a820-db15d6db45eb None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:41.727456 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:41.741100 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b61991-92e4-4ab5-ac4e-71120e32a97e None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:41.780665 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa34ca-b7a7-46dc-971d-05f93e93eb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:41.781615 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa34ca-b7a7-46dc-971d-05f93e93eb97 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:41.781980 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa34ca-b7a7-46dc-971d-05f93e93eb97 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:41.782277 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa34ca-b7a7-46dc-971d-05f93e93eb97 None admin] PATH_INFO: `/v3/projects/a83dc386fda8411eb2d4aef8e43b7065` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:41.784638 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa34ca-b7a7-46dc-971d-05f93e93eb97 None admin] RBAC: Authorizing `identity:get_project(project_id=a83dc386fda8411eb2d4aef8e43b7065)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:41.786162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa34ca-b7a7-46dc-971d-05f93e93eb97 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:41.795956 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 68/129] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:57:41 2022] GET /identity/v3/projects/a83dc386fda8411eb2d4aef8e43b7065 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:41.796793 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa34ca-b7a7-46dc-971d-05f93e93eb97 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:41.808010 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1c424-2a08-42b7-a820-db15d6db45eb None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:41.884439 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713dc9d-4cae-4241-a061-e00939bbfdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:41.885057 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713dc9d-4cae-4241-a061-e00939bbfdf7 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:41.885345 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713dc9d-4cae-4241-a061-e00939bbfdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:41.885626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713dc9d-4cae-4241-a061-e00939bbfdf7 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:41.886427 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713dc9d-4cae-4241-a061-e00939bbfdf7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:41.886825 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713dc9d-4cae-4241-a061-e00939bbfdf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:41.888277 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713dc9d-4cae-4241-a061-e00939bbfdf7 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:41.906047 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 62/130] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:57:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:41.907036 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713dc9d-4cae-4241-a061-e00939bbfdf7 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:41.920145 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:41.966756 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa34ca-b7a7-46dc-971d-05f93e93eb97 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:42.008344 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116de9e2-bc2f-4a3f-9074-d4883de9053b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:42.009243 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116de9e2-bc2f-4a3f-9074-d4883de9053b None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:42.009534 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116de9e2-bc2f-4a3f-9074-d4883de9053b None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:42.009808 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116de9e2-bc2f-4a3f-9074-d4883de9053b None admin] PATH_INFO: `/v3/users/26f5b3d45f5048aa9a49fb86df108d90` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:42.011823 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116de9e2-bc2f-4a3f-9074-d4883de9053b None admin] RBAC: Authorizing `identity:get_user(user_id=26f5b3d45f5048aa9a49fb86df108d90)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:42.015788 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116de9e2-bc2f-4a3f-9074-d4883de9053b None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:42.015917 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 69/131] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:57:41 2022] GET /identity/v3/users/26f5b3d45f5048aa9a49fb86df108d90 => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:42.016644 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116de9e2-bc2f-4a3f-9074-d4883de9053b None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:42.132779 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116de9e2-bc2f-4a3f-9074-d4883de9053b None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:42.159688 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:42.171485 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8146fc-46a5-4044-a9ff-1b59906039cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:42.171945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713dc9d-4cae-4241-a061-e00939bbfdf7 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:42.172389 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8146fc-46a5-4044-a9ff-1b59906039cf None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:42.172685 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8146fc-46a5-4044-a9ff-1b59906039cf None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:42.172968 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8146fc-46a5-4044-a9ff-1b59906039cf None admin] PATH_INFO: `/v3/users/9593d0f94eed4e3e83f1323aab2716a7` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:42.173883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:42.176493 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8146fc-46a5-4044-a9ff-1b59906039cf None admin] RBAC: Authorizing `identity:get_user(user_id=9593d0f94eed4e3e83f1323aab2716a7)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:42.178533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8146fc-46a5-4044-a9ff-1b59906039cf None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:42.180425 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 70/132] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:57:42 2022] GET /identity/v3/users/9593d0f94eed4e3e83f1323aab2716a7 => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:42.181047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8146fc-46a5-4044-a9ff-1b59906039cf None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:42.289000 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef5505-4a64-4f77-9514-820ba265afe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:42.289918 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef5505-4a64-4f77-9514-820ba265afe6 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:42.290231 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef5505-4a64-4f77-9514-820ba265afe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:42.290519 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef5505-4a64-4f77-9514-820ba265afe6 None admin] PATH_INFO: `/v3/projects/a83dc386fda8411eb2d4aef8e43b7065/users/26f5b3d45f5048aa9a49fb86df108d90/roles/e0b0c216ef51441ab34b6e036da09daa` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:42.292792 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ef5505-4a64-4f77-9514-820ba265afe6 None admin] RBAC: Authorizing `identity:create_grant(project_id=a83dc386fda8411eb2d4aef8e43b7065, user_id=26f5b3d45f5048aa9a49fb86df108d90, role_id=e0b0c216ef51441ab34b6e036da09daa)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:42.294523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ef5505-4a64-4f77-9514-820ba265afe6 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:42.297771 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8146fc-46a5-4044-a9ff-1b59906039cf None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:42.320345 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 63/133] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 15:57:41 2022] PUT /identity/v3/projects/a83dc386fda8411eb2d4aef8e43b7065/users/26f5b3d45f5048aa9a49fb86df108d90/roles/e0b0c216ef51441ab34b6e036da09daa => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:57:42.321277 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef5505-4a64-4f77-9514-820ba265afe6 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:42.341551 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4535e4-96ad-4190-aec7-9aa6f7791e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:42.343147 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4535e4-96ad-4190-aec7-9aa6f7791e77 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:42.343499 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4535e4-96ad-4190-aec7-9aa6f7791e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:42.343780 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4535e4-96ad-4190-aec7-9aa6f7791e77 None admin] PATH_INFO: `/v3/projects/c2db1b3048a147849ab9d1d46779d24b` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:42.345073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4535e4-96ad-4190-aec7-9aa6f7791e77 None admin] RBAC: Authorizing `identity:get_project(project_id=c2db1b3048a147849ab9d1d46779d24b)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:42.346674 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4535e4-96ad-4190-aec7-9aa6f7791e77 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:42.348913 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 71/134] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:57:42 2022] GET /identity/v3/projects/c2db1b3048a147849ab9d1d46779d24b => generated 312 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:42.355141 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4535e4-96ad-4190-aec7-9aa6f7791e77 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:42.445304 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:42.509159 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4535e4-96ad-4190-aec7-9aa6f7791e77 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:42.525703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef5505-4a64-4f77-9514-820ba265afe6 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:42.588724 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89bd71f-55fe-4b00-aa03-457eef8c3875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:42.589642 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89bd71f-55fe-4b00-aa03-457eef8c3875 None admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:42.589944 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89bd71f-55fe-4b00-aa03-457eef8c3875 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:42.593148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89bd71f-55fe-4b00-aa03-457eef8c3875 None admin] PATH_INFO: `/v3/projects/c2db1b3048a147849ab9d1d46779d24b/users/26f5b3d45f5048aa9a49fb86df108d90/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:42.595305 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89bd71f-55fe-4b00-aa03-457eef8c3875 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2db1b3048a147849ab9d1d46779d24b, user_id=26f5b3d45f5048aa9a49fb86df108d90, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:42.596971 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89bd71f-55fe-4b00-aa03-457eef8c3875 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:42.601510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad951df-6ad7-482b-96a0-473043e4bfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:42.602950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad951df-6ad7-482b-96a0-473043e4bfe5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:42.603249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad951df-6ad7-482b-96a0-473043e4bfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:42.603701 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad951df-6ad7-482b-96a0-473043e4bfe5 None admin] PATH_INFO: `/v3/projects/a83dc386fda8411eb2d4aef8e43b7065` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:42.607938 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad951df-6ad7-482b-96a0-473043e4bfe5 None admin] RBAC: Authorizing `identity:get_project(project_id=a83dc386fda8411eb2d4aef8e43b7065)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:42.609560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad951df-6ad7-482b-96a0-473043e4bfe5 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:42.611454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 64/135] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:57:42 2022] GET /identity/v3/projects/a83dc386fda8411eb2d4aef8e43b7065 => generated 298 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:42.618294 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad951df-6ad7-482b-96a0-473043e4bfe5 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:42.621165 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 72/136] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 15:57:42 2022] PUT /identity/v3/projects/c2db1b3048a147849ab9d1d46779d24b/users/26f5b3d45f5048aa9a49fb86df108d90/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:57:42.690290 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:42.730225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacb35e-2855-4e87-92ab-57575b66b0d3 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:42.730507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacb35e-2855-4e87-92ab-57575b66b0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:42.730711 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacb35e-2855-4e87-92ab-57575b66b0d3 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:42.732426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 73/137] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:57:42.738227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b974be-86c6-4ae9-8fc2-5bc57be59726 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:42.738524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b974be-86c6-4ae9-8fc2-5bc57be59726 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:42.738790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b974be-86c6-4ae9-8fc2-5bc57be59726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:42.798067 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-41b974be-86c6-4ae9-8fc2-5bc57be59726 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:42.799298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-41b974be-86c6-4ae9-8fc2-5bc57be59726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:42.895074 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad951df-6ad7-482b-96a0-473043e4bfe5 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:43.005333 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50cab7-ac1b-45bd-8f18-c7c3fa9025a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:43.006541 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50cab7-ac1b-45bd-8f18-c7c3fa9025a1 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:43.006541 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50cab7-ac1b-45bd-8f18-c7c3fa9025a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:43.006815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50cab7-ac1b-45bd-8f18-c7c3fa9025a1 None admin] PATH_INFO: `/v3/projects/a83dc386fda8411eb2d4aef8e43b7065/users/9593d0f94eed4e3e83f1323aab2716a7/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:43.008492 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 74/138] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:57:43.009106 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50cab7-ac1b-45bd-8f18-c7c3fa9025a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=a83dc386fda8411eb2d4aef8e43b7065, user_id=9593d0f94eed4e3e83f1323aab2716a7, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:43.010309 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbeb5bf-0703-4a48-a311-e081b6229825 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:43.010618 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50cab7-ac1b-45bd-8f18-c7c3fa9025a1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:43.010986 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbeb5bf-0703-4a48-a311-e081b6229825 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:43.011444 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbeb5bf-0703-4a48-a311-e081b6229825 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:43.012974 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 75/139] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:43.020262 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48225ff3-535d-41db-9650-8ceb1654210e None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:43.021359 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48225ff3-535d-41db-9650-8ceb1654210e None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:43.022076 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48225ff3-535d-41db-9650-8ceb1654210e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:43.036201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 65/140] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 15:57:42 2022] PUT /identity/v3/projects/a83dc386fda8411eb2d4aef8e43b7065/users/9593d0f94eed4e3e83f1323aab2716a7/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:57:43.042897 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50cab7-ac1b-45bd-8f18-c7c3fa9025a1 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:43.052656 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-48225ff3-535d-41db-9650-8ceb1654210e None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:43.054529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-48225ff3-535d-41db-9650-8ceb1654210e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:43.174627 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:43.212283 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50cab7-ac1b-45bd-8f18-c7c3fa9025a1 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:43.214109 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:43.268605 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 76/141] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:43.286243 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63192b7-fb14-478f-bc5c-acbca01754d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:43.287371 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63192b7-fb14-478f-bc5c-acbca01754d6 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:43.287662 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63192b7-fb14-478f-bc5c-acbca01754d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:43.287942 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63192b7-fb14-478f-bc5c-acbca01754d6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:43.304172 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63192b7-fb14-478f-bc5c-acbca01754d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:57:43.312719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-c63192b7-fb14-478f-bc5c-acbca01754d6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:57:43.312719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:43.312719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:43.312719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:43.312719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:43.312719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:43.312719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:43.312719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:43.312719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:43.312719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:43.312719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:43.312719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:43.312719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:57:43.312719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:57:43.312719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:57:43.312719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:57:43.313560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:57:43.313560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:57:43.313560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:57:43.313560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:57:43.313560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:43.313560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:43.313560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:43.313560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:43.313560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:43.313560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:43.313560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:43.313560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:43.313560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:43.313560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:43.313560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:43.313560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:43.314354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:43.314354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:43.314354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:43.314354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:43.314354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:57:43.314354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:57:43.314354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:57:43.314354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:57:43.314354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:57:43.314354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:57:43.314354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:57:43.314354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:57:43.314354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 66/142] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:57:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:57:43.321666 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48225ff3-535d-41db-9650-8ceb1654210e None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:43.328655 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63192b7-fb14-478f-bc5c-acbca01754d6 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:43.459262 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63192b7-fb14-478f-bc5c-acbca01754d6 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:43.526832 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e591e-4f23-4709-aab9-7af3129db6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:43.527613 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e591e-4f23-4709-aab9-7af3129db6d1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:43.528449 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e591e-4f23-4709-aab9-7af3129db6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:43.528781 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e591e-4f23-4709-aab9-7af3129db6d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:43.530014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e591e-4f23-4709-aab9-7af3129db6d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:43.531491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e591e-4f23-4709-aab9-7af3129db6d1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:43.534346 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 67/143] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:57:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:57:43.540357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e591e-4f23-4709-aab9-7af3129db6d1 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:43.584196 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:43.594704 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48225ff3-535d-41db-9650-8ceb1654210e None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:43.643663 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d046b1-f6cd-4373-bd4e-69722f8ca0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:43.644350 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d046b1-f6cd-4373-bd4e-69722f8ca0d4 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:43.644635 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d046b1-f6cd-4373-bd4e-69722f8ca0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:43.644911 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d046b1-f6cd-4373-bd4e-69722f8ca0d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:43.645673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d046b1-f6cd-4373-bd4e-69722f8ca0d4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:43.646006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d046b1-f6cd-4373-bd4e-69722f8ca0d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:43.647432 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d046b1-f6cd-4373-bd4e-69722f8ca0d4 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:43.656456 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 77/144] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 15:57:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:43.661889 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d046b1-f6cd-4373-bd4e-69722f8ca0d4 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:43.726304 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:43.758160 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e591e-4f23-4709-aab9-7af3129db6d1 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:43.800528 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d046b1-f6cd-4373-bd4e-69722f8ca0d4 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:43.836234 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c772c0-14e3-4784-98f4-2f129d6ce48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:43.837117 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c772c0-14e3-4784-98f4-2f129d6ce48d None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:43.837398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c772c0-14e3-4784-98f4-2f129d6ce48d None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:43.837669 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c772c0-14e3-4784-98f4-2f129d6ce48d None admin] PATH_INFO: `/v3/users/059108b974dc423a92557b010d5fbde9` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:43.838920 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c772c0-14e3-4784-98f4-2f129d6ce48d None admin] RBAC: Authorizing `identity:get_user(user_id=059108b974dc423a92557b010d5fbde9)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:43.841300 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c772c0-14e3-4784-98f4-2f129d6ce48d None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:43.842560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582a7a2-0483-4865-bc85-b8a2174e048c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:43.843252 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582a7a2-0483-4865-bc85-b8a2174e048c None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:43.843356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 78/145] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:57:43 2022] GET /identity/v3/users/059108b974dc423a92557b010d5fbde9 => generated 289 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:57:43.843539 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582a7a2-0483-4865-bc85-b8a2174e048c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:43.843812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582a7a2-0483-4865-bc85-b8a2174e048c None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:43.844544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e582a7a2-0483-4865-bc85-b8a2174e048c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:43.846015 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e582a7a2-0483-4865-bc85-b8a2174e048c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:43.848701 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c772c0-14e3-4784-98f4-2f129d6ce48d None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:43.895442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 68/146] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 15:57:43 2022] POST /identity/v3/users => generated 283 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:57:44.007832 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c772c0-14e3-4784-98f4-2f129d6ce48d None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:44.027494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:44.051198 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7a560-51b6-4478-9223-0a29a3ed90d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:44.052355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7a560-51b6-4478-9223-0a29a3ed90d4 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:44.052746 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7a560-51b6-4478-9223-0a29a3ed90d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:44.053122 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7a560-51b6-4478-9223-0a29a3ed90d4 None admin] PATH_INFO: `/v3/projects/a83dc386fda8411eb2d4aef8e43b7065` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:44.054515 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7a560-51b6-4478-9223-0a29a3ed90d4 None admin] RBAC: Authorizing `identity:get_project(project_id=a83dc386fda8411eb2d4aef8e43b7065)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:44.056411 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7a560-51b6-4478-9223-0a29a3ed90d4 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:44.062688 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 79/147] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:57:43 2022] GET /identity/v3/projects/a83dc386fda8411eb2d4aef8e43b7065 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:44.068771 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582a7a2-0483-4865-bc85-b8a2174e048c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:44.182016 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:44.224725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582a7a2-0483-4865-bc85-b8a2174e048c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:44.265473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a8555-6c88-43b4-9603-5c7894ddf228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:44.266474 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a8555-6c88-43b4-9603-5c7894ddf228 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:44.266924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a8555-6c88-43b4-9603-5c7894ddf228 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:44.267096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a8555-6c88-43b4-9603-5c7894ddf228 None admin] PATH_INFO: `/v3/projects/a83dc386fda8411eb2d4aef8e43b7065/users/059108b974dc423a92557b010d5fbde9/roles/1a524a543caa49b286168391bb19d71e` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:44.269327 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a8555-6c88-43b4-9603-5c7894ddf228 None admin] RBAC: Authorizing `identity:create_grant(project_id=a83dc386fda8411eb2d4aef8e43b7065, user_id=059108b974dc423a92557b010d5fbde9, role_id=1a524a543caa49b286168391bb19d71e)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:44.271058 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a8555-6c88-43b4-9603-5c7894ddf228 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:44.286368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 69/148] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 15:57:44 2022] PUT /identity/v3/projects/a83dc386fda8411eb2d4aef8e43b7065/users/059108b974dc423a92557b010d5fbde9/roles/1a524a543caa49b286168391bb19d71e => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:57:44.660736 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b497534-ca83-46ab-8b03-52ec410e4c1f None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:44.661231 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b497534-ca83-46ab-8b03-52ec410e4c1f None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:44.661304 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b497534-ca83-46ab-8b03-52ec410e4c1f None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:44.662712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 80/149] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:44.668326 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e67caa-5592-4fea-b9fa-046c52695967 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:44.669331 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e67caa-5592-4fea-b9fa-046c52695967 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:44.669536 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e67caa-5592-4fea-b9fa-046c52695967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:44.694993 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c9e67caa-5592-4fea-b9fa-046c52695967 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:44.696223 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c9e67caa-5592-4fea-b9fa-046c52695967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:44.794616 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 70/150] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:44.835440 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b497534-ca83-46ab-8b03-52ec410e4c1f None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:44.965112 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b497534-ca83-46ab-8b03-52ec410e4c1f None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:45.004873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f079a9-9243-46cf-aa3b-17313d618bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:45.006062 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f079a9-9243-46cf-aa3b-17313d618bba None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:45.006378 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f079a9-9243-46cf-aa3b-17313d618bba None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:45.006662 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f079a9-9243-46cf-aa3b-17313d618bba None admin] PATH_INFO: `/v3/roles/member` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:45.019409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f079a9-9243-46cf-aa3b-17313d618bba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:57:45.027578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-85f079a9-9243-46cf-aa3b-17313d618bba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:57:45.027578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:45.027578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:45.027578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:45.027578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:45.027578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:45.027578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:45.027578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:45.027578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:45.027578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:45.027578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:45.027578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:45.027578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:57:45.027578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:57:45.027578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:57:45.027578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:57:45.028932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:57:45.028932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:57:45.028932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:57:45.028932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:57:45.028932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:45.028932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:45.028932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:45.028932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:45.028932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:45.028932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:45.028932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:45.028932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:45.028932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:45.028932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:45.028932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:45.028932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:45.030337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:45.030337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:45.030337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:45.030337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:45.030337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:57:45.030337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:57:45.030337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:57:45.030337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:57:45.030337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:57:45.030337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:57:45.030337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:57:45.030337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:57:45.030337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 81/151] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:57:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:57:45.035448 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e67caa-5592-4fea-b9fa-046c52695967 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:45.114789 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:45.167462 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e67caa-5592-4fea-b9fa-046c52695967 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:45.211462 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702e2c6-a6ec-49d6-ba23-d40fd5773af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:45.212220 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702e2c6-a6ec-49d6-ba23-d40fd5773af8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:45.212630 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702e2c6-a6ec-49d6-ba23-d40fd5773af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:45.212920 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702e2c6-a6ec-49d6-ba23-d40fd5773af8 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:45.213797 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702e2c6-a6ec-49d6-ba23-d40fd5773af8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:45.214164 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702e2c6-a6ec-49d6-ba23-d40fd5773af8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:45.216538 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702e2c6-a6ec-49d6-ba23-d40fd5773af8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:45.230801 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 71/152] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 15:57:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:45.237307 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f079a9-9243-46cf-aa3b-17313d618bba None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:45.301009 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:45.374748 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f079a9-9243-46cf-aa3b-17313d618bba None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:45.417272 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854be9fe-2eb3-48d0-bfd6-c8a0e289013a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:45.418248 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854be9fe-2eb3-48d0-bfd6-c8a0e289013a None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:45.419327 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854be9fe-2eb3-48d0-bfd6-c8a0e289013a None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:45.419727 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854be9fe-2eb3-48d0-bfd6-c8a0e289013a None admin] PATH_INFO: `/v3/users/26f5b3d45f5048aa9a49fb86df108d90` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:45.421005 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854be9fe-2eb3-48d0-bfd6-c8a0e289013a None admin] RBAC: Authorizing `identity:get_user(user_id=26f5b3d45f5048aa9a49fb86df108d90)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:45.422518 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854be9fe-2eb3-48d0-bfd6-c8a0e289013a None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:45.424737 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 82/153] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:57:45 2022] GET /identity/v3/users/26f5b3d45f5048aa9a49fb86df108d90 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:45.430472 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702e2c6-a6ec-49d6-ba23-d40fd5773af8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:45.554305 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aeb4d4-9e18-4946-ba48-383b47f88524 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:45.554633 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aeb4d4-9e18-4946-ba48-383b47f88524 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:45.554909 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aeb4d4-9e18-4946-ba48-383b47f88524 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:45.556842 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 83/154] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:45.565083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702e2c6-a6ec-49d6-ba23-d40fd5773af8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:45.569610 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174c535-302d-4bbe-b6c1-2015db9e56f1 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:45.569905 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174c535-302d-4bbe-b6c1-2015db9e56f1 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:45.570243 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174c535-302d-4bbe-b6c1-2015db9e56f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:45.597182 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:45.598137 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-9174c535-302d-4bbe-b6c1-2015db9e56f1 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:45.599407 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-9174c535-302d-4bbe-b6c1-2015db9e56f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:45.629755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bfacb5-50c6-4a71-bd37-7aeab09fa720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:45.631144 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfacb5-50c6-4a71-bd37-7aeab09fa720 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:45.631653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfacb5-50c6-4a71-bd37-7aeab09fa720 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:45.632130 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfacb5-50c6-4a71-bd37-7aeab09fa720 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:45.762696 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 84/155] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:45.789716 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bfacb5-50c6-4a71-bd37-7aeab09fa720 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:45.791863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bfacb5-50c6-4a71-bd37-7aeab09fa720 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:45.807636 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9174c535-302d-4bbe-b6c1-2015db9e56f1 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:45.912804 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:45.923152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 72/156] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:57:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:57:45.929016 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bfacb5-50c6-4a71-bd37-7aeab09fa720 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:45.961796 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:46.061671 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9174c535-302d-4bbe-b6c1-2015db9e56f1 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:46.096218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bfacb5-50c6-4a71-bd37-7aeab09fa720 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:46.159977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c760c8-1d5d-46f6-a64e-81f796337370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:46.160788 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c760c8-1d5d-46f6-a64e-81f796337370 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:46.161073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c760c8-1d5d-46f6-a64e-81f796337370 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:46.161345 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c760c8-1d5d-46f6-a64e-81f796337370 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:46.178848 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ec1b1-ce6f-499d-bf2e-a06426d392d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:46.179755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ec1b1-ce6f-499d-bf2e-a06426d392d2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:46.180083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ec1b1-ce6f-499d-bf2e-a06426d392d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:46.180320 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ec1b1-ce6f-499d-bf2e-a06426d392d2 None admin] PATH_INFO: `/v3/projects/a83dc386fda8411eb2d4aef8e43b7065` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:46.181633 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ec1b1-ce6f-499d-bf2e-a06426d392d2 None admin] RBAC: Authorizing `identity:get_project(project_id=a83dc386fda8411eb2d4aef8e43b7065)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:46.183148 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ec1b1-ce6f-499d-bf2e-a06426d392d2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:46.185105 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 73/157] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:57:45 2022] GET /identity/v3/projects/a83dc386fda8411eb2d4aef8e43b7065 => generated 298 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:57:46.186111 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c760c8-1d5d-46f6-a64e-81f796337370 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:57:46.191187 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ec1b1-ce6f-499d-bf2e-a06426d392d2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:46.198395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-70c760c8-1d5d-46f6-a64e-81f796337370 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:57:46.198395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:46.198395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:46.198395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:46.198395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:46.198395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:46.198395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:46.198395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:46.198395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:46.198395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:46.198395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:46.198395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:46.198395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:57:46.198395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:57:46.198395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:57:46.198395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:57:46.199430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:57:46.199430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:57:46.199430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:57:46.199430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:57:46.199430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:46.199430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:46.199430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:46.199430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:46.199430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:46.199430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:46.199430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:46.199430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:46.199430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:46.199430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:46.199430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:46.199430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:46.200783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:46.200783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:46.200783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:46.200783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:46.200783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:57:46.200783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:57:46.200783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:57:46.200783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:57:46.200783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:57:46.200783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:57:46.200783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:57:46.200783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:57:46.200783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 85/158] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 15:57:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:57:46.211348 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c760c8-1d5d-46f6-a64e-81f796337370 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:46.336911 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:46.364844 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ec1b1-ce6f-499d-bf2e-a06426d392d2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:46.408014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150d41e-436f-4ebd-8f63-3307103b55bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:46.408921 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150d41e-436f-4ebd-8f63-3307103b55bf None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:46.409463 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150d41e-436f-4ebd-8f63-3307103b55bf None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:46.410779 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150d41e-436f-4ebd-8f63-3307103b55bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:46.412528 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d150d41e-436f-4ebd-8f63-3307103b55bf None admin] RBAC: Adding query filter params (role.id=3ce570490865493380e6941d004136bd, scope.project.id=a83dc386fda8411eb2d4aef8e43b7065, user.id=26f5b3d45f5048aa9a49fb86df108d90) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:46.413192 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d150d41e-436f-4ebd-8f63-3307103b55bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:46.415962 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d150d41e-436f-4ebd-8f63-3307103b55bf None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:46.431942 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 74/159] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:57:46 2022] GET /identity/v3/role_assignments?user.id=26f5b3d45f5048aa9a49fb86df108d90&scope.project.id=a83dc386fda8411eb2d4aef8e43b7065&role.id=3ce570490865493380e6941d004136bd => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:57:46.435090 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d67d3-9e00-410e-b12f-0085be6abd99 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:46.435936 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d67d3-9e00-410e-b12f-0085be6abd99 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:46.437739 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d67d3-9e00-410e-b12f-0085be6abd99 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:46.440238 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 75/160] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:46.445056 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70795a-1a87-41c1-8376-bf999e156c01 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:46.445529 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70795a-1a87-41c1-8376-bf999e156c01 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:46.446653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70795a-1a87-41c1-8376-bf999e156c01 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:46.448375 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 76/161] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:46.450823 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c760c8-1d5d-46f6-a64e-81f796337370 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:46.453902 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd394c0-a397-44b8-9c15-bd404871082b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:46.454780 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd394c0-a397-44b8-9c15-bd404871082b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:46.455904 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:46.456815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd394c0-a397-44b8-9c15-bd404871082b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:46.489465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ccd394c0-a397-44b8-9c15-bd404871082b None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:46.494739 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ccd394c0-a397-44b8-9c15-bd404871082b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:46.517944 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ece86-f5f2-4fe3-bc18-2b923107181f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:46.517944 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ece86-f5f2-4fe3-bc18-2b923107181f None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:46.517944 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ece86-f5f2-4fe3-bc18-2b923107181f None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:46.517944 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ece86-f5f2-4fe3-bc18-2b923107181f None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:46.518335 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043ece86-f5f2-4fe3-bc18-2b923107181f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:46.518335 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043ece86-f5f2-4fe3-bc18-2b923107181f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:46.518335 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043ece86-f5f2-4fe3-bc18-2b923107181f None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:46.525497 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 86/162] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:57:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:57:46.526600 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237731f0-52e6-44de-ae03-eafb7b66457c None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:46.526890 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237731f0-52e6-44de-ae03-eafb7b66457c None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:46.527272 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237731f0-52e6-44de-ae03-eafb7b66457c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:46.545647 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-237731f0-52e6-44de-ae03-eafb7b66457c None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:46.546886 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-237731f0-52e6-44de-ae03-eafb7b66457c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:46.655664 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 87/163] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:46.656486 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237731f0-52e6-44de-ae03-eafb7b66457c None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:46.661027 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 77/164] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:46.704856 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd394c0-a397-44b8-9c15-bd404871082b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:46.832201 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:46.845363 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237731f0-52e6-44de-ae03-eafb7b66457c None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:46.906330 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf049a0-207d-4ce1-b310-6a0bd1258de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:46.907147 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf049a0-207d-4ce1-b310-6a0bd1258de6 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:46.907432 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf049a0-207d-4ce1-b310-6a0bd1258de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:46.907711 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf049a0-207d-4ce1-b310-6a0bd1258de6 None admin] PATH_INFO: `/v3/users/26f5b3d45f5048aa9a49fb86df108d90` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:46.908948 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf049a0-207d-4ce1-b310-6a0bd1258de6 None admin] RBAC: Authorizing `identity:get_user(user_id=26f5b3d45f5048aa9a49fb86df108d90)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:46.910472 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf049a0-207d-4ce1-b310-6a0bd1258de6 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:46.913082 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 88/165] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:57:46 2022] GET /identity/v3/users/26f5b3d45f5048aa9a49fb86df108d90 => generated 275 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:46.913891 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf049a0-207d-4ce1-b310-6a0bd1258de6 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:46.927753 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:46.976173 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd394c0-a397-44b8-9c15-bd404871082b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:47.052078 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c64bd-e8c0-43bd-ac50-ab72571a74c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:47.052896 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c64bd-e8c0-43bd-ac50-ab72571a74c8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:47.053184 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c64bd-e8c0-43bd-ac50-ab72571a74c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:47.053466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c64bd-e8c0-43bd-ac50-ab72571a74c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:47.071514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c64bd-e8c0-43bd-ac50-ab72571a74c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:57:47.086454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-8e9c64bd-e8c0-43bd-ac50-ab72571a74c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:57:47.086454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:47.086454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:47.086454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:47.086454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:47.086454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:47.086454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:47.086454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:47.086454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:47.086454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:47.086454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:47.086454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:47.086454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:57:47.086454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:57:47.086454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:57:47.086454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:57:47.087400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:57:47.087400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:57:47.087400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:57:47.087400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:57:47.087400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:47.087400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:47.087400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:47.087400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:47.087400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:47.087400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:47.087400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:47.087400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:47.087400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:47.087400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:47.087400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:47.087400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:47.088157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:47.088157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:47.088157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:47.088157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:47.088157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:57:47.088157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:57:47.088157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:57:47.088157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:57:47.088157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:57:47.088157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:57:47.088157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:57:47.088157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:57:47.088157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 78/166] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 15:57:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:57:47.093482 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c64bd-e8c0-43bd-ac50-ab72571a74c8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:47.169580 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf049a0-207d-4ce1-b310-6a0bd1258de6 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:47.247650 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c64bd-e8c0-43bd-ac50-ab72571a74c8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:47.264007 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680aaa05-ab41-4e96-a918-aa1e11f26cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:47.264834 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680aaa05-ab41-4e96-a918-aa1e11f26cae None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:47.265117 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680aaa05-ab41-4e96-a918-aa1e11f26cae None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:47.265391 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680aaa05-ab41-4e96-a918-aa1e11f26cae None admin] PATH_INFO: `/v3/roles/member` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:47.281609 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680aaa05-ab41-4e96-a918-aa1e11f26cae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:57:47.293125 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3e6c9-c1d7-4d93-8cde-e5f28530b2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:47.293858 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3e6c9-c1d7-4d93-8cde-e5f28530b2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:47.294173 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3e6c9-c1d7-4d93-8cde-e5f28530b2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:47.294453 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3e6c9-c1d7-4d93-8cde-e5f28530b2f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:47.301130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-680aaa05-ab41-4e96-a918-aa1e11f26cae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:57:47.301130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:47.301130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:47.301130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:47.301130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:47.301130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:47.301130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:47.301130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:47.301130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:47.301130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:47.301130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:47.301130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:47.301130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:57:47.301130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:57:47.301130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:57:47.301130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:57:47.303225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:57:47.303225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:57:47.303225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:57:47.303225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:57:47.303225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:47.303225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:47.303225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:47.303225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:47.303225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:47.303225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:47.303225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:47.303225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:47.303225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:47.303225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:47.303225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:47.303225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:47.305114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:47.305114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:47.305114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:47.305114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:47.305114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:57:47.305114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:57:47.305114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:57:47.305114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:57:47.305114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:57:47.305114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:57:47.305114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:57:47.305114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:57:47.305114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 89/167] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:57:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:57:47.305114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680aaa05-ab41-4e96-a918-aa1e11f26cae None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:47.318543 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:47.361205 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:47.369086 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e3e6c9-c1d7-4d93-8cde-e5f28530b2f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:47.370622 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e3e6c9-c1d7-4d93-8cde-e5f28530b2f2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:47.466564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680aaa05-ab41-4e96-a918-aa1e11f26cae None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:47.479360 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 79/168] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:57:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:57:47.484944 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3e6c9-c1d7-4d93-8cde-e5f28530b2f2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:47.534520 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09365b4b-2d9c-43cc-90aa-dd487e845e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:47.535206 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09365b4b-2d9c-43cc-90aa-dd487e845e41 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:47.535493 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09365b4b-2d9c-43cc-90aa-dd487e845e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:47.535774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09365b4b-2d9c-43cc-90aa-dd487e845e41 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:47.536685 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09365b4b-2d9c-43cc-90aa-dd487e845e41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:47.537010 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09365b4b-2d9c-43cc-90aa-dd487e845e41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:47.538464 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09365b4b-2d9c-43cc-90aa-dd487e845e41 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:47.547103 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:47.558869 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 90/169] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:57:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:47.559883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09365b4b-2d9c-43cc-90aa-dd487e845e41 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:47.607734 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3e6c9-c1d7-4d93-8cde-e5f28530b2f2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:47.643162 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae049d1-f0e7-4547-bad2-65d3b6f05926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:47.643837 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae049d1-f0e7-4547-bad2-65d3b6f05926 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:47.644122 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae049d1-f0e7-4547-bad2-65d3b6f05926 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:47.644399 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae049d1-f0e7-4547-bad2-65d3b6f05926 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:47.645167 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae049d1-f0e7-4547-bad2-65d3b6f05926 None admin] RBAC: Adding query filter params (name=member) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:47.645473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae049d1-f0e7-4547-bad2-65d3b6f05926 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:47.646942 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae049d1-f0e7-4547-bad2-65d3b6f05926 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:47.657695 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 80/170] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 15:57:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:47.658688 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae049d1-f0e7-4547-bad2-65d3b6f05926 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:47.777414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae049d1-f0e7-4547-bad2-65d3b6f05926 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:47.786279 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:47.796804 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:47.816496 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957611f-1f64-4443-872e-ba981fcf0d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:47.817384 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957611f-1f64-4443-872e-ba981fcf0d82 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:47.817670 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957611f-1f64-4443-872e-ba981fcf0d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:47.817965 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957611f-1f64-4443-872e-ba981fcf0d82 None admin] PATH_INFO: `/v3/users/9593d0f94eed4e3e83f1323aab2716a7` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:47.819427 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9957611f-1f64-4443-872e-ba981fcf0d82 None admin] RBAC: Authorizing `identity:get_user(user_id=9593d0f94eed4e3e83f1323aab2716a7)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:47.821156 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9957611f-1f64-4443-872e-ba981fcf0d82 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:47.823631 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 81/171] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:57:47 2022] GET /identity/v3/users/9593d0f94eed4e3e83f1323aab2716a7 => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:47.824689 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957611f-1f64-4443-872e-ba981fcf0d82 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:47.840014 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09365b4b-2d9c-43cc-90aa-dd487e845e41 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:47.937550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957611f-1f64-4443-872e-ba981fcf0d82 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:47.976865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebd017-4472-474c-acad-b2eafa69e1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:47.977746 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebd017-4472-474c-acad-b2eafa69e1d5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:47.978159 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebd017-4472-474c-acad-b2eafa69e1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:47.978445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebd017-4472-474c-acad-b2eafa69e1d5 None admin] PATH_INFO: `/v3/users/26f5b3d45f5048aa9a49fb86df108d90` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:47.979719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ebd017-4472-474c-acad-b2eafa69e1d5 None admin] RBAC: Authorizing `identity:get_user(user_id=26f5b3d45f5048aa9a49fb86df108d90)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:47.981259 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ebd017-4472-474c-acad-b2eafa69e1d5 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:47.982948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 82/172] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:57:47 2022] GET /identity/v3/users/26f5b3d45f5048aa9a49fb86df108d90 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:47.983818 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebd017-4472-474c-acad-b2eafa69e1d5 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:47.986797 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f3184-4606-4631-bcec-e405d53aa19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:47.987760 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f3184-4606-4631-bcec-e405d53aa19b None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:47.988052 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f3184-4606-4631-bcec-e405d53aa19b None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:47.988330 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f3184-4606-4631-bcec-e405d53aa19b None admin] PATH_INFO: `/v3/projects/a83dc386fda8411eb2d4aef8e43b7065` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:47.990775 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f3184-4606-4631-bcec-e405d53aa19b None admin] RBAC: Authorizing `identity:get_project(project_id=a83dc386fda8411eb2d4aef8e43b7065)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:47.992508 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f3184-4606-4631-bcec-e405d53aa19b None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:47.994567 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 91/173] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:57:47 2022] GET /identity/v3/projects/a83dc386fda8411eb2d4aef8e43b7065 => generated 298 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:47.995563 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f3184-4606-4631-bcec-e405d53aa19b None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:48.042545 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:48.111055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebd017-4472-474c-acad-b2eafa69e1d5 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:48.151546 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ebf0e3-c07b-4c5d-9fad-546112bbd1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:48.152212 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebf0e3-c07b-4c5d-9fad-546112bbd1fa None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:48.152499 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebf0e3-c07b-4c5d-9fad-546112bbd1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:48.152779 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebf0e3-c07b-4c5d-9fad-546112bbd1fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:48.232393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ebf0e3-c07b-4c5d-9fad-546112bbd1fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:48.234132 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ebf0e3-c07b-4c5d-9fad-546112bbd1fa None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:48.268333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f3184-4606-4631-bcec-e405d53aa19b None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:48.297221 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:48.353806 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca54229-17d6-4e4f-9bb8-496fcfb006cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:48.354570 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca54229-17d6-4e4f-9bb8-496fcfb006cc None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:48.354911 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca54229-17d6-4e4f-9bb8-496fcfb006cc None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:48.355198 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca54229-17d6-4e4f-9bb8-496fcfb006cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:48.382438 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:48.393343 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 83/174] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:57:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:57:48.394447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ebf0e3-c07b-4c5d-9fad-546112bbd1fa None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:48.431365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca54229-17d6-4e4f-9bb8-496fcfb006cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:48.433199 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca54229-17d6-4e4f-9bb8-496fcfb006cc None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:48.548078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 92/175] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:57:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:57:48.549850 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9ae50-6614-4afa-bdca-5b3d76fc2ff8 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:48.550372 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9ae50-6614-4afa-bdca-5b3d76fc2ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:48.550879 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9ae50-6614-4afa-bdca-5b3d76fc2ff8 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:48.556553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 93/176] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:48.557650 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d9ae50-6614-4afa-bdca-5b3d76fc2ff8 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:48.656979 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ebf0e3-c07b-4c5d-9fad-546112bbd1fa None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:48.677325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:48.720222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a0f39-bb47-4fbd-b8ae-c48c03a4d7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:48.721285 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a0f39-bb47-4fbd-b8ae-c48c03a4d7fc None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:48.722891 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a0f39-bb47-4fbd-b8ae-c48c03a4d7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:48.724343 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a0f39-bb47-4fbd-b8ae-c48c03a4d7fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:48.725396 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2a0f39-bb47-4fbd-b8ae-c48c03a4d7fc None admin] RBAC: Adding query filter params (role.id=e0b0c216ef51441ab34b6e036da09daa, scope.project.id=a83dc386fda8411eb2d4aef8e43b7065, user.id=26f5b3d45f5048aa9a49fb86df108d90) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:48.726850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2a0f39-bb47-4fbd-b8ae-c48c03a4d7fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:48.729453 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2a0f39-bb47-4fbd-b8ae-c48c03a4d7fc None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:48.743296 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 84/177] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:57:48 2022] GET /identity/v3/role_assignments?user.id=26f5b3d45f5048aa9a49fb86df108d90&scope.project.id=a83dc386fda8411eb2d4aef8e43b7065&role.id=e0b0c216ef51441ab34b6e036da09daa => generated 608 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:48.747453 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a0f39-bb47-4fbd-b8ae-c48c03a4d7fc None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:48.764874 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:48.790651 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d9ae50-6614-4afa-bdca-5b3d76fc2ff8 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:48.886907 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a0f39-bb47-4fbd-b8ae-c48c03a4d7fc None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:48.898791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3767aa3-64ea-4f94-87b5-32110ac0c45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:48.900313 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3767aa3-64ea-4f94-87b5-32110ac0c45c None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:48.900746 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3767aa3-64ea-4f94-87b5-32110ac0c45c None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:48.901185 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3767aa3-64ea-4f94-87b5-32110ac0c45c None admin] PATH_INFO: `/v3/projects/a83dc386fda8411eb2d4aef8e43b7065` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:48.902804 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3767aa3-64ea-4f94-87b5-32110ac0c45c None admin] RBAC: Authorizing `identity:get_project(project_id=a83dc386fda8411eb2d4aef8e43b7065)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:48.905788 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3767aa3-64ea-4f94-87b5-32110ac0c45c None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:48.908005 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 94/178] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:57:48 2022] GET /identity/v3/projects/a83dc386fda8411eb2d4aef8e43b7065 => generated 298 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:48.909581 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bb999-3e52-4b84-b4cb-81c8197f8662 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:48.918191 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bb999-3e52-4b84-b4cb-81c8197f8662 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:48.918869 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bb999-3e52-4b84-b4cb-81c8197f8662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:48.938946 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0772ccd2-1cfc-4695-8f5b-9047345faf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:48.939839 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772ccd2-1cfc-4695-8f5b-9047345faf89 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:48.940122 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772ccd2-1cfc-4695-8f5b-9047345faf89 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:48.940397 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772ccd2-1cfc-4695-8f5b-9047345faf89 None admin] PATH_INFO: `/v3/projects/c2db1b3048a147849ab9d1d46779d24b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:48.941815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0772ccd2-1cfc-4695-8f5b-9047345faf89 None admin] RBAC: Authorizing `identity:get_project(project_id=c2db1b3048a147849ab9d1d46779d24b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:48.945355 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0772ccd2-1cfc-4695-8f5b-9047345faf89 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:48.948808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 85/179] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:57:48 2022] GET /identity/v3/projects/c2db1b3048a147849ab9d1d46779d24b => generated 312 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:48.951308 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0772ccd2-1cfc-4695-8f5b-9047345faf89 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:48.953540 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-0d0bb999-3e52-4b84-b4cb-81c8197f8662 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:48.954852 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-0d0bb999-3e52-4b84-b4cb-81c8197f8662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:49.079371 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:49.081849 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 95/180] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:49.083017 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0bb999-3e52-4b84-b4cb-81c8197f8662 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:49.120049 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0772ccd2-1cfc-4695-8f5b-9047345faf89 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:49.142088 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:49.165609 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe4c84-c46f-4254-816a-f32c6153f9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:49.166444 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe4c84-c46f-4254-816a-f32c6153f9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:49.166742 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe4c84-c46f-4254-816a-f32c6153f9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:49.167004 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe4c84-c46f-4254-816a-f32c6153f9f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:49.167860 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe4c84-c46f-4254-816a-f32c6153f9f0 None admin] RBAC: Adding query filter params (role.id=bbbc77ec645d4ff5a8bf29ce0377f578, scope.project.id=a83dc386fda8411eb2d4aef8e43b7065, user.id=9593d0f94eed4e3e83f1323aab2716a7) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:49.168172 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe4c84-c46f-4254-816a-f32c6153f9f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:49.169890 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe4c84-c46f-4254-816a-f32c6153f9f0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:49.185823 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 86/181] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:57:48 2022] GET /identity/v3/role_assignments?user.id=9593d0f94eed4e3e83f1323aab2716a7&scope.project.id=a83dc386fda8411eb2d4aef8e43b7065&role.id=bbbc77ec645d4ff5a8bf29ce0377f578 => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:49.185823 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372c5bf-2f0a-40f7-b388-6358bcd33a7d None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:49.185823 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372c5bf-2f0a-40f7-b388-6358bcd33a7d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:49.185823 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372c5bf-2f0a-40f7-b388-6358bcd33a7d None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:49.186665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 87/182] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:49.187960 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974615dc-78a8-4157-8d4e-747207ad0dda None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:49.188245 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974615dc-78a8-4157-8d4e-747207ad0dda None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:49.188521 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974615dc-78a8-4157-8d4e-747207ad0dda None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:49.190038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 88/183] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:49.194872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974615dc-78a8-4157-8d4e-747207ad0dda None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:49.219609 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0bb999-3e52-4b84-b4cb-81c8197f8662 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:49.264682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339dc838-25ce-497f-8d43-4135a203d2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:49.265366 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339dc838-25ce-497f-8d43-4135a203d2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:49.265653 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339dc838-25ce-497f-8d43-4135a203d2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:49.265942 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339dc838-25ce-497f-8d43-4135a203d2f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:49.266966 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339dc838-25ce-497f-8d43-4135a203d2f0 None admin] RBAC: Adding query filter params (role.id=3ce570490865493380e6941d004136bd, scope.project.id=c2db1b3048a147849ab9d1d46779d24b, user.id=26f5b3d45f5048aa9a49fb86df108d90) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:49.267273 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339dc838-25ce-497f-8d43-4135a203d2f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:49.268869 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339dc838-25ce-497f-8d43-4135a203d2f0 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:49.275869 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 96/184] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:57:49 2022] GET /identity/v3/role_assignments?user.id=26f5b3d45f5048aa9a49fb86df108d90&scope.project.id=c2db1b3048a147849ab9d1d46779d24b&role.id=3ce570490865493380e6941d004136bd => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:49.277686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911cd0f3-7f94-4dc5-a661-dcc4412e2be3 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:49.277686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911cd0f3-7f94-4dc5-a661-dcc4412e2be3 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:49.278038 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911cd0f3-7f94-4dc5-a661-dcc4412e2be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:49.301643 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-911cd0f3-7f94-4dc5-a661-dcc4412e2be3 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:49.303375 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-911cd0f3-7f94-4dc5-a661-dcc4412e2be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:49.336423 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974615dc-78a8-4157-8d4e-747207ad0dda None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:49.389174 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2f04a-766e-4866-bf3e-2ebf1424bdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:49.389701 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 97/185] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:49.390509 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2f04a-766e-4866-bf3e-2ebf1424bdb3 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:49.390802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2f04a-766e-4866-bf3e-2ebf1424bdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:49.391074 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2f04a-766e-4866-bf3e-2ebf1424bdb3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:49.391360 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2fa63-99a1-4d32-86f9-2cfdfc1ea085 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:49.391701 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2fa63-99a1-4d32-86f9-2cfdfc1ea085 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:49.391945 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2fa63-99a1-4d32-86f9-2cfdfc1ea085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:49.412932 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:49.413114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-76a2fa63-99a1-4d32-86f9-2cfdfc1ea085 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:49.414214 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-76a2fa63-99a1-4d32-86f9-2cfdfc1ea085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:49.427325 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f2f04a-766e-4866-bf3e-2ebf1424bdb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:57:49.436208 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-a0f2f04a-766e-4866-bf3e-2ebf1424bdb3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:57:49.436208 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:49.436208 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:49.436208 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:49.436208 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:49.436208 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:49.436208 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:49.436208 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:49.436208 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:49.436208 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:49.436208 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:49.436208 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:49.436208 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:57:49.436208 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:57:49.436208 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:57:49.436208 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:57:49.437145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:57:49.437145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:57:49.437145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:57:49.437145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:57:49.437145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:49.437145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:49.437145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:49.437145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:49.437145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:49.437145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:49.437145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:49.437145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:49.437145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:49.437145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:49.437145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:49.437145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:49.438450 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:49.438450 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:49.438450 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:49.438450 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:49.438450 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:57:49.438450 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:57:49.438450 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:57:49.438450 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:57:49.438450 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:57:49.438450 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:57:49.438450 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:57:49.438450 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:57:49.438450 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 89/186] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:57:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:57:49.439471 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aad777-0173-442d-842e-8ae90f8bfa86 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:49.439538 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aad777-0173-442d-842e-8ae90f8bfa86 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:49.439973 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aad777-0173-442d-842e-8ae90f8bfa86 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:49.441179 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 90/187] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:49.442427 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aad777-0173-442d-842e-8ae90f8bfa86 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:49.473732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:49.508252 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 98/188] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:49.510644 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2fa63-99a1-4d32-86f9-2cfdfc1ea085 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:49.581326 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aad777-0173-442d-842e-8ae90f8bfa86 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:49.616383 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdda1d8-3f59-4c4d-8e70-05b87a08196f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:49.617181 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdda1d8-3f59-4c4d-8e70-05b87a08196f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:49.617459 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdda1d8-3f59-4c4d-8e70-05b87a08196f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:49.617728 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdda1d8-3f59-4c4d-8e70-05b87a08196f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:49.628227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdda1d8-3f59-4c4d-8e70-05b87a08196f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:57:49.628899 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2fa63-99a1-4d32-86f9-2cfdfc1ea085 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:49.636087 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-dcdda1d8-3f59-4c4d-8e70-05b87a08196f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:57:49.636087 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:49.636087 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:49.636087 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:49.636087 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:49.636087 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:49.636087 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:49.636087 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:49.636087 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:49.636087 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:49.636087 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:49.636087 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:49.636087 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:57:49.636087 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:57:49.636087 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:57:49.636087 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:57:49.636982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:57:49.636982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:57:49.636982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:57:49.636982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:57:49.636982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:49.636982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:49.636982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:49.636982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:49.636982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:49.636982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:49.636982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:49.636982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:49.636982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:49.636982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:49.636982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:49.636982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:49.637806 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:49.637806 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:49.637806 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:49.637806 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:49.637806 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:57:49.637806 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:57:49.637806 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:57:49.637806 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:57:49.637806 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:57:49.637806 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:57:49.637806 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:57:49.637806 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:57:49.637806 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 91/189] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 15:57:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:57:49.639178 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27c0d8-0df3-4a74-b2a2-5fef6ca05380 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:49.639178 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27c0d8-0df3-4a74-b2a2-5fef6ca05380 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:49.639427 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27c0d8-0df3-4a74-b2a2-5fef6ca05380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:49.656119 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-bf27c0d8-0df3-4a74-b2a2-5fef6ca05380 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:49.657312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bf27c0d8-0df3-4a74-b2a2-5fef6ca05380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:49.666202 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1567b8-cac5-4e32-8e20-eb94e98125ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:49.666867 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1567b8-cac5-4e32-8e20-eb94e98125ee None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:49.667167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1567b8-cac5-4e32-8e20-eb94e98125ee None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:49.667436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1567b8-cac5-4e32-8e20-eb94e98125ee None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:49.668188 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1567b8-cac5-4e32-8e20-eb94e98125ee None admin] RBAC: Adding query filter params (name=reader) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:49.668498 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1567b8-cac5-4e32-8e20-eb94e98125ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:49.670500 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1567b8-cac5-4e32-8e20-eb94e98125ee None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:49.678883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 99/190] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 15:57:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:49.679597 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1567b8-cac5-4e32-8e20-eb94e98125ee None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:49.715159 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:49.722702 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:49.732183 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 92/191] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:49.733222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27c0d8-0df3-4a74-b2a2-5fef6ca05380 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:49.793025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1567b8-cac5-4e32-8e20-eb94e98125ee None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:49.831886 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41ea54-45fc-4b8c-960a-de3c4c9e37d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:49.832689 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41ea54-45fc-4b8c-960a-de3c4c9e37d4 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:49.832969 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41ea54-45fc-4b8c-960a-de3c4c9e37d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:49.833246 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41ea54-45fc-4b8c-960a-de3c4c9e37d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:49.834525 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb41ea54-45fc-4b8c-960a-de3c4c9e37d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:49.835989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb41ea54-45fc-4b8c-960a-de3c4c9e37d4 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:49.837629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 100/192] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:57:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:49.838432 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41ea54-45fc-4b8c-960a-de3c4c9e37d4 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:49.852293 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27c0d8-0df3-4a74-b2a2-5fef6ca05380 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:49.889526 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da74840-e542-4b4c-bac8-83295ea569a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:49.890241 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da74840-e542-4b4c-bac8-83295ea569a9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:49.890527 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da74840-e542-4b4c-bac8-83295ea569a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:49.890803 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da74840-e542-4b4c-bac8-83295ea569a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:49.891559 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da74840-e542-4b4c-bac8-83295ea569a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:49.891867 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da74840-e542-4b4c-bac8-83295ea569a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:49.893281 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da74840-e542-4b4c-bac8-83295ea569a9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:49.901520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 93/193] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:57:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:49.902351 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da74840-e542-4b4c-bac8-83295ea569a9 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:49.945336 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41ea54-45fc-4b8c-960a-de3c4c9e37d4 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:49.952950 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:49.953286 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:49.981033 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44e143-60bc-4416-9ad7-f40ea2429514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:49.982006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44e143-60bc-4416-9ad7-f40ea2429514 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:49.982298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44e143-60bc-4416-9ad7-f40ea2429514 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:49.982636 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44e143-60bc-4416-9ad7-f40ea2429514 None admin] PATH_INFO: `/v3/users/059108b974dc423a92557b010d5fbde9` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:49.985106 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d44e143-60bc-4416-9ad7-f40ea2429514 None admin] RBAC: Authorizing `identity:get_user(user_id=059108b974dc423a92557b010d5fbde9)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:49.986642 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d44e143-60bc-4416-9ad7-f40ea2429514 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:49.988310 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 101/194] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:57:49 2022] GET /identity/v3/users/059108b974dc423a92557b010d5fbde9 => generated 289 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:49.989057 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44e143-60bc-4416-9ad7-f40ea2429514 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:50.005141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da74840-e542-4b4c-bac8-83295ea569a9 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:50.039838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03056a20-5900-42aa-8333-04d3b12839ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:50.040648 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03056a20-5900-42aa-8333-04d3b12839ad None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:50.040930 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03056a20-5900-42aa-8333-04d3b12839ad None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:50.041204 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03056a20-5900-42aa-8333-04d3b12839ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:50.052000 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03056a20-5900-42aa-8333-04d3b12839ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:57:50.060070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-03056a20-5900-42aa-8333-04d3b12839ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:57:50.060070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:50.060070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:50.060070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:50.060070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:50.060070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:50.060070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:50.060070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:50.060070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:50.060070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:50.060070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:50.060070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:50.060070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:57:50.060070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:57:50.060070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:57:50.060070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:57:50.060948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:57:50.060948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:57:50.060948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:57:50.060948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:57:50.060948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:50.060948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:50.060948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:50.060948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:50.060948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:50.060948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:50.060948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:50.060948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:50.060948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:50.060948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:50.060948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:50.060948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:50.061733 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:50.061733 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:50.061733 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:50.061733 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:50.061733 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:57:50.061733 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:57:50.061733 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:57:50.061733 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:57:50.061733 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:57:50.061733 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:57:50.061733 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:57:50.061733 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:57:50.061733 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 94/195] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 15:57:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:57:50.062430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03056a20-5900-42aa-8333-04d3b12839ad None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:50.095613 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44e143-60bc-4416-9ad7-f40ea2429514 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:50.130219 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7857b8-178a-4463-9f29-52baa1c30b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:50.131141 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7857b8-178a-4463-9f29-52baa1c30b98 None admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:50.131420 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7857b8-178a-4463-9f29-52baa1c30b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:50.131692 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7857b8-178a-4463-9f29-52baa1c30b98 None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:50.132404 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7857b8-178a-4463-9f29-52baa1c30b98 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:50.133861 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7857b8-178a-4463-9f29-52baa1c30b98 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:50.156110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 102/196] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 15:57:49 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:57:50.159368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7857b8-178a-4463-9f29-52baa1c30b98 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:50.171186 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03056a20-5900-42aa-8333-04d3b12839ad None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:50.184679 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:50.199230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:50.215063 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef13175-845e-41c0-8570-34688e0171f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:50.216562 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef13175-845e-41c0-8570-34688e0171f0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:50.217348 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef13175-845e-41c0-8570-34688e0171f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:50.217348 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef13175-845e-41c0-8570-34688e0171f0 None admin] PATH_INFO: `/v3/users/71e063c2f41c4034a33858116dcb702f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:50.233695 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef13175-845e-41c0-8570-34688e0171f0 None admin] RBAC: Authorizing `identity:get_user(user_id=71e063c2f41c4034a33858116dcb702f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:50.235291 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef13175-845e-41c0-8570-34688e0171f0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:50.237249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 95/197] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:57:50 2022] GET /identity/v3/users/71e063c2f41c4034a33858116dcb702f => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:50.237968 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef13175-845e-41c0-8570-34688e0171f0 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:50.293235 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7857b8-178a-4463-9f29-52baa1c30b98 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:50.334609 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e8758-d705-40c3-bb04-c803dd68d366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:50.335474 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e8758-d705-40c3-bb04-c803dd68d366 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:50.335583 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e8758-d705-40c3-bb04-c803dd68d366 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:50.335830 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e8758-d705-40c3-bb04-c803dd68d366 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:50.382807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef13175-845e-41c0-8570-34688e0171f0 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:50.418287 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e8758-d705-40c3-bb04-c803dd68d366 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:50.420459 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e8758-d705-40c3-bb04-c803dd68d366 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:50.431110 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d108f-c3b9-4cfb-980b-f9d22ebea1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:50.431801 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d108f-c3b9-4cfb-980b-f9d22ebea1f2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:50.432085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d108f-c3b9-4cfb-980b-f9d22ebea1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:50.432362 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d108f-c3b9-4cfb-980b-f9d22ebea1f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:50.433123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d108f-c3b9-4cfb-980b-f9d22ebea1f2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:50.433431 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d108f-c3b9-4cfb-980b-f9d22ebea1f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:50.435790 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d108f-c3b9-4cfb-980b-f9d22ebea1f2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:50.446340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 96/198] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:57:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:50.448266 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d108f-c3b9-4cfb-980b-f9d22ebea1f2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:50.462047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:50.487720 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:50.555854 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 103/199] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:57:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:57:50.556823 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e8758-d705-40c3-bb04-c803dd68d366 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:50.569891 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d108f-c3b9-4cfb-980b-f9d22ebea1f2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:50.607565 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69318196-ba36-48c2-b922-cfb25c2aa9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:50.608241 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69318196-ba36-48c2-b922-cfb25c2aa9d8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:50.608528 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69318196-ba36-48c2-b922-cfb25c2aa9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:50.608810 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69318196-ba36-48c2-b922-cfb25c2aa9d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:50.684768 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e8758-d705-40c3-bb04-c803dd68d366 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:50.702863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69318196-ba36-48c2-b922-cfb25c2aa9d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:50.704630 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69318196-ba36-48c2-b922-cfb25c2aa9d8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:50.717789 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:50.725088 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f8b3e-2256-4445-8233-d3327dc54f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:50.726086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f8b3e-2256-4445-8233-d3327dc54f30 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:50.726432 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f8b3e-2256-4445-8233-d3327dc54f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:50.726773 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f8b3e-2256-4445-8233-d3327dc54f30 None admin] PATH_INFO: `/v3/users/71e063c2f41c4034a33858116dcb702f` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:50.728075 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f8b3e-2256-4445-8233-d3327dc54f30 None admin] RBAC: Authorizing `identity:get_user(user_id=71e063c2f41c4034a33858116dcb702f)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:50.729695 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f8b3e-2256-4445-8233-d3327dc54f30 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:50.731888 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 104/200] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:57:50 2022] GET /identity/v3/users/71e063c2f41c4034a33858116dcb702f => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:50.736791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f8b3e-2256-4445-8233-d3327dc54f30 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:50.746866 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:50.847928 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 97/201] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:57:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:57:50.848799 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69318196-ba36-48c2-b922-cfb25c2aa9d8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:50.870351 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f8b3e-2256-4445-8233-d3327dc54f30 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:50.925991 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c1a07-55af-432e-a9d9-63884bf18d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:50.927396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c1a07-55af-432e-a9d9-63884bf18d95 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:50.927689 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c1a07-55af-432e-a9d9-63884bf18d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:50.928257 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c1a07-55af-432e-a9d9-63884bf18d95 None admin] PATH_INFO: `/v3/projects/a83dc386fda8411eb2d4aef8e43b7065` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:50.929778 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c1a07-55af-432e-a9d9-63884bf18d95 None admin] RBAC: Authorizing `identity:get_project(project_id=a83dc386fda8411eb2d4aef8e43b7065)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:50.931936 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c1a07-55af-432e-a9d9-63884bf18d95 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:50.938088 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 105/202] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:57:50 2022] GET /identity/v3/projects/a83dc386fda8411eb2d4aef8e43b7065 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:50.938957 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c1a07-55af-432e-a9d9-63884bf18d95 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:50.976948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69318196-ba36-48c2-b922-cfb25c2aa9d8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:51.021603 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:51.024269 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:51.031716 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8374942-2240-4ed7-9e3e-d45abfb1a2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:51.033362 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8374942-2240-4ed7-9e3e-d45abfb1a2e4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:51.033652 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8374942-2240-4ed7-9e3e-d45abfb1a2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:51.033955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8374942-2240-4ed7-9e3e-d45abfb1a2e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:51.064673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c1a07-55af-432e-a9d9-63884bf18d95 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:51.107880 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ba477-7998-4024-a225-a77649a9c9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:51.108790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ba477-7998-4024-a225-a77649a9c9b3 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:51.109074 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ba477-7998-4024-a225-a77649a9c9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:51.109350 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ba477-7998-4024-a225-a77649a9c9b3 None admin] PATH_INFO: `/v3/projects/328b693aa62e4467aa34a749006ad00f` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:51.110621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81ba477-7998-4024-a225-a77649a9c9b3 None admin] RBAC: Authorizing `identity:get_project(project_id=328b693aa62e4467aa34a749006ad00f)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:51.112102 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81ba477-7998-4024-a225-a77649a9c9b3 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:51.118385 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 106/203] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:57:50 2022] GET /identity/v3/projects/328b693aa62e4467aa34a749006ad00f => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:51.119275 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ba477-7998-4024-a225-a77649a9c9b3 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:51.120212 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8374942-2240-4ed7-9e3e-d45abfb1a2e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:51.121788 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8374942-2240-4ed7-9e3e-d45abfb1a2e4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:51.241562 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ba477-7998-4024-a225-a77649a9c9b3 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:51.250826 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 98/204] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:57:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:57:51.256739 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8374942-2240-4ed7-9e3e-d45abfb1a2e4 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:51.289172 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:51.291521 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae41a13-35ee-4b6b-b877-4af2a6219700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:51.293052 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae41a13-35ee-4b6b-b877-4af2a6219700 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:51.293512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae41a13-35ee-4b6b-b877-4af2a6219700 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:51.294034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae41a13-35ee-4b6b-b877-4af2a6219700 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:51.295054 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae41a13-35ee-4b6b-b877-4af2a6219700 None admin] RBAC: Adding query filter params (role.id=1a524a543caa49b286168391bb19d71e, scope.project.id=a83dc386fda8411eb2d4aef8e43b7065, user.id=059108b974dc423a92557b010d5fbde9) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:51.295899 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae41a13-35ee-4b6b-b877-4af2a6219700 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:51.298516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae41a13-35ee-4b6b-b877-4af2a6219700 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:51.300682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:51.314355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 107/205] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:57:51 2022] GET /identity/v3/role_assignments?user.id=059108b974dc423a92557b010d5fbde9&scope.project.id=a83dc386fda8411eb2d4aef8e43b7065&role.id=1a524a543caa49b286168391bb19d71e => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:51.314355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae41a13-35ee-4b6b-b877-4af2a6219700 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:51.436432 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8374942-2240-4ed7-9e3e-d45abfb1a2e4 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:51.514959 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3703d36e-541f-40d2-b045-8e622b10adab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:51.515551 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703d36e-541f-40d2-b045-8e622b10adab None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:51.515832 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703d36e-541f-40d2-b045-8e622b10adab None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:51.516104 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703d36e-541f-40d2-b045-8e622b10adab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:51.516945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3703d36e-541f-40d2-b045-8e622b10adab None admin] RBAC: Adding query filter params (role.id=bbbc77ec645d4ff5a8bf29ce0377f578, scope.project.id=328b693aa62e4467aa34a749006ad00f, user.id=71e063c2f41c4034a33858116dcb702f) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:51.517257 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3703d36e-541f-40d2-b045-8e622b10adab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:51.518775 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3703d36e-541f-40d2-b045-8e622b10adab None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:51.522831 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae41a13-35ee-4b6b-b877-4af2a6219700 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:51.535689 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 99/206] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:57:51 2022] GET /identity/v3/role_assignments?user.id=71e063c2f41c4034a33858116dcb702f&scope.project.id=328b693aa62e4467aa34a749006ad00f&role.id=bbbc77ec645d4ff5a8bf29ce0377f578 => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:51.577050 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a73c73d-903e-4f32-8863-8d6d4fc85ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:51.578398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73c73d-903e-4f32-8863-8d6d4fc85ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:51.578914 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73c73d-903e-4f32-8863-8d6d4fc85ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:51.579387 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73c73d-903e-4f32-8863-8d6d4fc85ae6 None admin] PATH_INFO: `/v3/projects/328b693aa62e4467aa34a749006ad00f` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:51.580927 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a73c73d-903e-4f32-8863-8d6d4fc85ae6 None admin] RBAC: Authorizing `identity:get_project(project_id=328b693aa62e4467aa34a749006ad00f)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:51.582689 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a73c73d-903e-4f32-8863-8d6d4fc85ae6 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:51.584864 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 108/207] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:57:51 2022] GET /identity/v3/projects/328b693aa62e4467aa34a749006ad00f => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:51.592134 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3703d36e-541f-40d2-b045-8e622b10adab None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:51.706847 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:51.715216 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3703d36e-541f-40d2-b045-8e622b10adab None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:51.753626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ba8aa-7a2a-4f32-b473-aa41fa8717a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:51.755337 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ba8aa-7a2a-4f32-b473-aa41fa8717a8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:51.755337 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ba8aa-7a2a-4f32-b473-aa41fa8717a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:51.755337 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ba8aa-7a2a-4f32-b473-aa41fa8717a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:51.756443 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ba8aa-7a2a-4f32-b473-aa41fa8717a8 None admin] RBAC: Adding query filter params (role.id=e0b0c216ef51441ab34b6e036da09daa, scope.project.id=328b693aa62e4467aa34a749006ad00f, user.id=71e063c2f41c4034a33858116dcb702f) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:51.756830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ba8aa-7a2a-4f32-b473-aa41fa8717a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:51.758387 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ba8aa-7a2a-4f32-b473-aa41fa8717a8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:51.765211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 100/208] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:57:51 2022] GET /identity/v3/role_assignments?user.id=71e063c2f41c4034a33858116dcb702f&scope.project.id=328b693aa62e4467aa34a749006ad00f&role.id=e0b0c216ef51441ab34b6e036da09daa => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:52.532820 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99a486-906b-451a-9d6b-f907ca8537de None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:52.533349 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99a486-906b-451a-9d6b-f907ca8537de None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:52.533349 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99a486-906b-451a-9d6b-f907ca8537de None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:52.535082 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 109/209] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:52.540753 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad71de9-2e43-49d2-b062-f9b07f6acb96 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:52.541628 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad71de9-2e43-49d2-b062-f9b07f6acb96 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:52.542154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad71de9-2e43-49d2-b062-f9b07f6acb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:52.561173 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8ad71de9-2e43-49d2-b062-f9b07f6acb96 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:52.562768 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8ad71de9-2e43-49d2-b062-f9b07f6acb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:52.571535 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd2f91-0266-403f-b230-7ac70a10eef3 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:52.571817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd2f91-0266-403f-b230-7ac70a10eef3 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:52.572094 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd2f91-0266-403f-b230-7ac70a10eef3 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:52.573711 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 110/210] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:57:52.580739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a38a2d-0fd2-4e04-8ae2-b64ce342b8f2 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:52.581308 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a38a2d-0fd2-4e04-8ae2-b64ce342b8f2 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:52.581817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a38a2d-0fd2-4e04-8ae2-b64ce342b8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:52.600141 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-59a38a2d-0fd2-4e04-8ae2-b64ce342b8f2 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:52.601669 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-59a38a2d-0fd2-4e04-8ae2-b64ce342b8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:52.652484 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 101/211] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:52.675484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:52.682538 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad71de9-2e43-49d2-b062-f9b07f6acb96 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:52.686035 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 111/212] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:57:52.711862 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a38a2d-0fd2-4e04-8ae2-b64ce342b8f2 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:52.802906 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:52.819436 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad71de9-2e43-49d2-b062-f9b07f6acb96 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:52.842304 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a38a2d-0fd2-4e04-8ae2-b64ce342b8f2 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:52.865592 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd0a35-dd43-4b2f-b4b9-5ad5b64a86b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:52.867045 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd0a35-dd43-4b2f-b4b9-5ad5b64a86b9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:52.867532 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd0a35-dd43-4b2f-b4b9-5ad5b64a86b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:52.868028 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd0a35-dd43-4b2f-b4b9-5ad5b64a86b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:52.882373 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd0a35-dd43-4b2f-b4b9-5ad5b64a86b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:57:52.885492 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70764fe-a9fc-4534-8cd2-793f1eb99f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:52.886512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70764fe-a9fc-4534-8cd2-793f1eb99f07 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:52.887068 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70764fe-a9fc-4534-8cd2-793f1eb99f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:52.887514 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70764fe-a9fc-4534-8cd2-793f1eb99f07 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:52.889322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70764fe-a9fc-4534-8cd2-793f1eb99f07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:52.891040 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70764fe-a9fc-4534-8cd2-793f1eb99f07 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:52.892048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-cfbd0a35-dd43-4b2f-b4b9-5ad5b64a86b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:57:52.892048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:52.892048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:52.892048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:52.892048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:52.892048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:52.892048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:52.892048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:52.892048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:52.892048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:52.892048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:52.892048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:52.892048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:57:52.892048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:57:52.892048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:57:52.892048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:57:52.892970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:57:52.892970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:57:52.892970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:57:52.892970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:57:52.892970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:52.892970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:52.892970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:52.892970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:52.892970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:52.892970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:52.892970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:52.892970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:52.892970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:52.892970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:52.892970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:52.892970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:52.893753 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:52.893753 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:52.893753 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:52.893753 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:52.893753 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:57:52.893753 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:57:52.893753 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:57:52.893753 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:57:52.893753 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:57:52.893753 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:57:52.893753 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:57:52.893753 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:57:52.894489 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 102/213] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:57:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:57:52.895603 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 112/214] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:57:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:57:52.901011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd0a35-dd43-4b2f-b4b9-5ad5b64a86b9 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:52.903139 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70764fe-a9fc-4534-8cd2-793f1eb99f07 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:52.973512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:53.015670 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70764fe-a9fc-4534-8cd2-793f1eb99f07 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:53.017303 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd0a35-dd43-4b2f-b4b9-5ad5b64a86b9 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:53.054192 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694721a-24f6-4193-b11d-7b80e398f8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:53.054975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694721a-24f6-4193-b11d-7b80e398f8cf None admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:53.055261 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694721a-24f6-4193-b11d-7b80e398f8cf None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:53.055512 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711254b5-792c-4365-9284-ec5fb9d75aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:53.055626 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694721a-24f6-4193-b11d-7b80e398f8cf None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:53.056308 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694721a-24f6-4193-b11d-7b80e398f8cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:53.056638 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711254b5-792c-4365-9284-ec5fb9d75aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:53.057086 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711254b5-792c-4365-9284-ec5fb9d75aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:53.057554 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711254b5-792c-4365-9284-ec5fb9d75aa5 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:53.057830 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694721a-24f6-4193-b11d-7b80e398f8cf None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:53.058681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711254b5-792c-4365-9284-ec5fb9d75aa5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:53.059157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711254b5-792c-4365-9284-ec5fb9d75aa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:53.060797 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711254b5-792c-4365-9284-ec5fb9d75aa5 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:53.071844 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 103/215] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 15:57:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:53.081982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711254b5-792c-4365-9284-ec5fb9d75aa5 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:53.088955 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 113/216] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 15:57:52 2022] POST /identity/v3/users => generated 297 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:57:53.093774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:53.204664 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711254b5-792c-4365-9284-ec5fb9d75aa5 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:53.244187 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ed598-48bb-43e0-bebf-e4e7f3afc181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:53.245091 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ed598-48bb-43e0-bebf-e4e7f3afc181 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:53.245375 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ed598-48bb-43e0-bebf-e4e7f3afc181 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:53.245648 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ed598-48bb-43e0-bebf-e4e7f3afc181 None admin] PATH_INFO: `/v3/users/3cbb634531974782aee895d70907f60c` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:53.262197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ed598-48bb-43e0-bebf-e4e7f3afc181 None admin] RBAC: Authorizing `identity:get_user(user_id=3cbb634531974782aee895d70907f60c)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:53.263973 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ed598-48bb-43e0-bebf-e4e7f3afc181 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:53.266059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 104/217] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:57:53 2022] GET /identity/v3/users/3cbb634531974782aee895d70907f60c => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:53.271808 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694721a-24f6-4193-b11d-7b80e398f8cf None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:53.389608 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694721a-24f6-4193-b11d-7b80e398f8cf None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:53.413152 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:53.431892 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df2cc4-0c6f-4b25-b8a5-8e708be8f868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:53.432574 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df2cc4-0c6f-4b25-b8a5-8e708be8f868 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:53.432860 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df2cc4-0c6f-4b25-b8a5-8e708be8f868 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:53.433194 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df2cc4-0c6f-4b25-b8a5-8e708be8f868 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:53.521759 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df2cc4-0c6f-4b25-b8a5-8e708be8f868 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:53.523339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df2cc4-0c6f-4b25-b8a5-8e708be8f868 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:53.643690 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 114/218] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:57:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:57:53.649724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ed598-48bb-43e0-bebf-e4e7f3afc181 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:53.759962 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:53.804664 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ed598-48bb-43e0-bebf-e4e7f3afc181 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:53.812714 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f41c85-08d1-4b72-a472-e831e435a777 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:53.813037 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f41c85-08d1-4b72-a472-e831e435a777 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:53.813289 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f41c85-08d1-4b72-a472-e831e435a777 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:53.814989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 115/219] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:53.820572 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6cdfe7-f337-44a3-a211-1952e54e7748 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:53.820864 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6cdfe7-f337-44a3-a211-1952e54e7748 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:53.821153 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6cdfe7-f337-44a3-a211-1952e54e7748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:53.843223 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a83efe-b18d-42d5-9a37-aa43f5fa6f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:53.848935 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-aa6cdfe7-f337-44a3-a211-1952e54e7748 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:53.849080 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a83efe-b18d-42d5-9a37-aa43f5fa6f77 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:53.849274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a83efe-b18d-42d5-9a37-aa43f5fa6f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:53.849540 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a83efe-b18d-42d5-9a37-aa43f5fa6f77 None admin] PATH_INFO: `/v3/projects/328b693aa62e4467aa34a749006ad00f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:53.850208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-aa6cdfe7-f337-44a3-a211-1952e54e7748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:53.850879 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a83efe-b18d-42d5-9a37-aa43f5fa6f77 None admin] RBAC: Authorizing `identity:get_project(project_id=328b693aa62e4467aa34a749006ad00f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:53.852448 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a83efe-b18d-42d5-9a37-aa43f5fa6f77 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:53.858041 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 105/220] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:57:53 2022] GET /identity/v3/projects/328b693aa62e4467aa34a749006ad00f => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:53.861437 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a83efe-b18d-42d5-9a37-aa43f5fa6f77 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:53.870312 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:53.948491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 116/221] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:53.983009 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6cdfe7-f337-44a3-a211-1952e54e7748 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:54.046825 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a83efe-b18d-42d5-9a37-aa43f5fa6f77 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:54.088875 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ba1ab-73d8-42ec-add9-27f981494b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:54.089544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ba1ab-73d8-42ec-add9-27f981494b8b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:54.089826 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ba1ab-73d8-42ec-add9-27f981494b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:54.090130 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ba1ab-73d8-42ec-add9-27f981494b8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:54.091760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ba1ab-73d8-42ec-add9-27f981494b8b None admin] RBAC: Adding query filter params (role.id=3ce570490865493380e6941d004136bd, scope.project.id=328b693aa62e4467aa34a749006ad00f, user.id=3cbb634531974782aee895d70907f60c) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:54.092115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ba1ab-73d8-42ec-add9-27f981494b8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:54.093964 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ba1ab-73d8-42ec-add9-27f981494b8b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:54.099536 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:54.105854 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 106/222] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:57:53 2022] GET /identity/v3/role_assignments?user.id=3cbb634531974782aee895d70907f60c&scope.project.id=328b693aa62e4467aa34a749006ad00f&role.id=3ce570490865493380e6941d004136bd => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:57:54.107796 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e97800-be3a-476e-86f2-371acab85861 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:54.108589 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e97800-be3a-476e-86f2-371acab85861 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:54.108881 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e97800-be3a-476e-86f2-371acab85861 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:54.111785 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 107/223] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:54.116597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc357bc-74a3-4936-8ac6-2460bb55dd77 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:54.116882 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc357bc-74a3-4936-8ac6-2460bb55dd77 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:54.117173 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc357bc-74a3-4936-8ac6-2460bb55dd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:54.126401 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6cdfe7-f337-44a3-a211-1952e54e7748 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:54.147112 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-dbc357bc-74a3-4936-8ac6-2460bb55dd77 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:54.149126 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-dbc357bc-74a3-4936-8ac6-2460bb55dd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:54.235592 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 108/224] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:54.239243 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5eeeed-4b0d-4f64-9402-95e387d55331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:54.240132 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5eeeed-4b0d-4f64-9402-95e387d55331 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:54.240481 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5eeeed-4b0d-4f64-9402-95e387d55331 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:54.240827 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5eeeed-4b0d-4f64-9402-95e387d55331 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:54.254356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5eeeed-4b0d-4f64-9402-95e387d55331 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:57:54.262511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-3e5eeeed-4b0d-4f64-9402-95e387d55331 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:57:54.262511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:54.262511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:54.262511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:54.262511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:54.262511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:54.262511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:54.262511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:54.262511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:54.262511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:54.262511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:54.262511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:54.262511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:57:54.262511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:57:54.262511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:57:54.262511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:57:54.265060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:57:54.265060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:57:54.265060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:57:54.265060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:57:54.265060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:54.265060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:54.265060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:54.265060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:54.265060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:54.265060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:54.265060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:54.265060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:54.265060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:54.265060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:54.265060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:54.265060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:54.267130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:54.267130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:54.267130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:54.267130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:54.267130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:57:54.267130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:57:54.267130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:57:54.267130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:57:54.267130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:57:54.267130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:57:54.267130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:57:54.267130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:57:54.267130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 117/225] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 15:57:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:57:54.274490 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc357bc-74a3-4936-8ac6-2460bb55dd77 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:54.285301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5eeeed-4b0d-4f64-9402-95e387d55331 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:54.314192 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:54.410275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc357bc-74a3-4936-8ac6-2460bb55dd77 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:54.443170 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:54.452866 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126b2ba-d8b8-4ba2-a9b2-56d672c9d46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:54.453549 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126b2ba-d8b8-4ba2-a9b2-56d672c9d46d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:54.453831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126b2ba-d8b8-4ba2-a9b2-56d672c9d46d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:54.454136 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126b2ba-d8b8-4ba2-a9b2-56d672c9d46d None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:54.454897 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0126b2ba-d8b8-4ba2-a9b2-56d672c9d46d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:54.455205 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0126b2ba-d8b8-4ba2-a9b2-56d672c9d46d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:54.458186 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0126b2ba-d8b8-4ba2-a9b2-56d672c9d46d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:54.463782 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5eeeed-4b0d-4f64-9402-95e387d55331 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:54.483945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 109/226] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:57:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:54.489858 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126b2ba-d8b8-4ba2-a9b2-56d672c9d46d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:54.541961 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8fdc3b-7642-44be-a0d3-9601080c8d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:54.542779 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8fdc3b-7642-44be-a0d3-9601080c8d01 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:54.543065 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8fdc3b-7642-44be-a0d3-9601080c8d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:54.543344 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8fdc3b-7642-44be-a0d3-9601080c8d01 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:54.586011 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8fdc3b-7642-44be-a0d3-9601080c8d01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:57:54.594268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-cc8fdc3b-7642-44be-a0d3-9601080c8d01 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:57:54.594268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:54.594268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:54.594268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:54.594268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:54.594268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:54.594268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:54.594268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:54.594268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:54.594268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:54.594268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:54.594268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:54.594268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:57:54.594268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:57:54.594268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:57:54.594268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:57:54.595140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:57:54.595140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:57:54.595140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:57:54.595140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:57:54.595140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:54.595140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:54.595140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:54.595140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:54.595140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:54.595140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:54.595140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:54.595140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:54.595140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:54.595140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:54.595140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:54.595140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:54.595938 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:54.595938 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:54.595938 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:54.595938 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:54.595938 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:57:54.595938 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:57:54.595938 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:57:54.595938 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:57:54.595938 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:57:54.595938 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:57:54.595938 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:57:54.595938 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:57:54.595938 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 118/227] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 15:57:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:57:54.607909 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8fdc3b-7642-44be-a0d3-9601080c8d01 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:54.643133 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126b2ba-d8b8-4ba2-a9b2-56d672c9d46d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:54.689162 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb538cb6-0954-4b3a-b9bd-93484d4ab23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:54.690926 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb538cb6-0954-4b3a-b9bd-93484d4ab23f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:54.691414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb538cb6-0954-4b3a-b9bd-93484d4ab23f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:54.691891 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb538cb6-0954-4b3a-b9bd-93484d4ab23f None admin] PATH_INFO: `/v3/users/71e063c2f41c4034a33858116dcb702f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:54.697750 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb538cb6-0954-4b3a-b9bd-93484d4ab23f None admin] RBAC: Authorizing `identity:get_user(user_id=71e063c2f41c4034a33858116dcb702f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:54.699605 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb538cb6-0954-4b3a-b9bd-93484d4ab23f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:54.701640 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 110/228] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:57:54 2022] GET /identity/v3/users/71e063c2f41c4034a33858116dcb702f => generated 283 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:54.710945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb538cb6-0954-4b3a-b9bd-93484d4ab23f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:54.770369 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:54.797894 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:54.817140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8fdc3b-7642-44be-a0d3-9601080c8d01 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:54.857712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660424d-79dc-48bf-83dc-b20a3e4b7ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:54.858959 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660424d-79dc-48bf-83dc-b20a3e4b7ccd None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:54.859557 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660424d-79dc-48bf-83dc-b20a3e4b7ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:54.860082 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660424d-79dc-48bf-83dc-b20a3e4b7ccd None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:54.861329 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3660424d-79dc-48bf-83dc-b20a3e4b7ccd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:54.861843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3660424d-79dc-48bf-83dc-b20a3e4b7ccd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:54.863486 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3660424d-79dc-48bf-83dc-b20a3e4b7ccd None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:54.875421 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 119/229] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:57:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:54.881800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660424d-79dc-48bf-83dc-b20a3e4b7ccd None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:54.904915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb538cb6-0954-4b3a-b9bd-93484d4ab23f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:55.006949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660424d-79dc-48bf-83dc-b20a3e4b7ccd None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:55.020057 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925b9a2-4915-4663-9105-3a5cf80b7d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:55.020994 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925b9a2-4915-4663-9105-3a5cf80b7d6d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:55.021218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925b9a2-4915-4663-9105-3a5cf80b7d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:55.021497 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925b9a2-4915-4663-9105-3a5cf80b7d6d None admin] PATH_INFO: `/v3/projects/328b693aa62e4467aa34a749006ad00f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:55.026699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3925b9a2-4915-4663-9105-3a5cf80b7d6d None admin] RBAC: Authorizing `identity:get_project(project_id=328b693aa62e4467aa34a749006ad00f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:55.028187 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3925b9a2-4915-4663-9105-3a5cf80b7d6d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:55.030169 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 111/230] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:57:54 2022] GET /identity/v3/projects/328b693aa62e4467aa34a749006ad00f => generated 302 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:55.038055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925b9a2-4915-4663-9105-3a5cf80b7d6d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:55.050560 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:55.063052 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4044b-9704-423c-ac31-9266e61177e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:55.065339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4044b-9704-423c-ac31-9266e61177e0 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:55.065909 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4044b-9704-423c-ac31-9266e61177e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:55.066634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4044b-9704-423c-ac31-9266e61177e0 None admin] PATH_INFO: `/v3/users/71e063c2f41c4034a33858116dcb702f` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:55.069246 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4044b-9704-423c-ac31-9266e61177e0 None admin] RBAC: Authorizing `identity:get_user(user_id=71e063c2f41c4034a33858116dcb702f)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:55.072077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4044b-9704-423c-ac31-9266e61177e0 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:55.077397 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 120/231] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:57:54 2022] GET /identity/v3/users/71e063c2f41c4034a33858116dcb702f => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:55.081750 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4044b-9704-423c-ac31-9266e61177e0 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:55.257204 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4044b-9704-423c-ac31-9266e61177e0 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:55.259414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925b9a2-4915-4663-9105-3a5cf80b7d6d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:55.317499 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:55.330208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc965a-8823-4fd6-801e-f59a21eb21fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:55.331866 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc965a-8823-4fd6-801e-f59a21eb21fe None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:55.332798 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc965a-8823-4fd6-801e-f59a21eb21fe None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:55.333607 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc965a-8823-4fd6-801e-f59a21eb21fe None admin] PATH_INFO: `/v3/projects/328b693aa62e4467aa34a749006ad00f` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:55.335458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84214fdf-f161-468c-b41b-d9d062fd7272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:55.336570 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84214fdf-f161-468c-b41b-d9d062fd7272 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:55.337445 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc965a-8823-4fd6-801e-f59a21eb21fe None admin] RBAC: Authorizing `identity:get_project(project_id=328b693aa62e4467aa34a749006ad00f)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:55.337959 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84214fdf-f161-468c-b41b-d9d062fd7272 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:55.338439 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84214fdf-f161-468c-b41b-d9d062fd7272 None admin] PATH_INFO: `/v3/projects/328b693aa62e4467aa34a749006ad00f/users/71e063c2f41c4034a33858116dcb702f/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:55.340138 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc965a-8823-4fd6-801e-f59a21eb21fe None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:55.343093 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 121/232] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:57:55 2022] GET /identity/v3/projects/328b693aa62e4467aa34a749006ad00f => generated 302 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:55.344665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84214fdf-f161-468c-b41b-d9d062fd7272 None admin] RBAC: Authorizing `identity:create_grant(project_id=328b693aa62e4467aa34a749006ad00f, user_id=71e063c2f41c4034a33858116dcb702f, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:55.346477 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84214fdf-f161-468c-b41b-d9d062fd7272 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:55.349715 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc965a-8823-4fd6-801e-f59a21eb21fe None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:55.371539 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 112/233] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 15:57:55 2022] PUT /identity/v3/projects/328b693aa62e4467aa34a749006ad00f/users/71e063c2f41c4034a33858116dcb702f/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:57:55.438094 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:55.476734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc965a-8823-4fd6-801e-f59a21eb21fe None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:55.518028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdde421-0c65-4806-8ebc-2f5e502a47b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:55.519297 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdde421-0c65-4806-8ebc-2f5e502a47b0 None admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:55.519789 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdde421-0c65-4806-8ebc-2f5e502a47b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:55.520268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdde421-0c65-4806-8ebc-2f5e502a47b0 None admin] PATH_INFO: `/v3/projects/328b693aa62e4467aa34a749006ad00f/users/71e063c2f41c4034a33858116dcb702f/roles/e0b0c216ef51441ab34b6e036da09daa` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:55.522970 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdde421-0c65-4806-8ebc-2f5e502a47b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=328b693aa62e4467aa34a749006ad00f, user_id=71e063c2f41c4034a33858116dcb702f, role_id=e0b0c216ef51441ab34b6e036da09daa)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:55.524834 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdde421-0c65-4806-8ebc-2f5e502a47b0 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:55.545171 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 122/234] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 15:57:55 2022] PUT /identity/v3/projects/328b693aa62e4467aa34a749006ad00f/users/71e063c2f41c4034a33858116dcb702f/roles/e0b0c216ef51441ab34b6e036da09daa => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:57:55.894135 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d804521-a294-4312-9327-101289973075 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:55.895217 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d804521-a294-4312-9327-101289973075 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:55.895739 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d804521-a294-4312-9327-101289973075 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:55.897480 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 113/235] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:55.904209 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2fab0-5e2e-4992-860e-0e51de14030f None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:55.905195 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2fab0-5e2e-4992-860e-0e51de14030f None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:55.905745 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2fab0-5e2e-4992-860e-0e51de14030f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:55.925592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-e3f2fab0-5e2e-4992-860e-0e51de14030f None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:55.927173 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-e3f2fab0-5e2e-4992-860e-0e51de14030f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:55.963198 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51c926-aeed-4819-949e-7be289c5a4dd None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:55.963504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51c926-aeed-4819-949e-7be289c5a4dd None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:55.963759 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51c926-aeed-4819-949e-7be289c5a4dd None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:55.965406 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 114/236] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:55.973947 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7aea40-efd0-4a9f-a619-0bea87efd326 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:55.973947 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7aea40-efd0-4a9f-a619-0bea87efd326 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:55.973947 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7aea40-efd0-4a9f-a619-0bea87efd326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:55.991720 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ba7aea40-efd0-4a9f-a619-0bea87efd326 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:55.993536 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ba7aea40-efd0-4a9f-a619-0bea87efd326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:56.030671 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 123/237] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:56.054894 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f2fab0-5e2e-4992-860e-0e51de14030f None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:56.103888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 115/238] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:57:56.128950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7aea40-efd0-4a9f-a619-0bea87efd326 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:56.150150 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:56.231950 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f2fab0-5e2e-4992-860e-0e51de14030f None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:56.269434 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1afb65-5afb-49ad-a7cf-5577776c1a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:56.270363 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1afb65-5afb-49ad-a7cf-5577776c1a1e None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:56.270716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1afb65-5afb-49ad-a7cf-5577776c1a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:56.271058 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1afb65-5afb-49ad-a7cf-5577776c1a1e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:56.282637 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1afb65-5afb-49ad-a7cf-5577776c1a1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:57:56.290908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-7e1afb65-5afb-49ad-a7cf-5577776c1a1e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:57:56.290908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:56.290908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:56.290908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:56.290908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:56.290908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:56.290908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:56.290908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:56.290908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:56.290908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:56.290908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:56.290908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:56.290908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:57:56.290908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:57:56.290908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:57:56.290908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:57:56.292025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:57:56.292025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:57:56.292025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:57:56.292025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:57:56.292025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:56.292025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:56.292025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:56.292025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:56.292025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:56.292025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:56.292025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:56.292025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:56.292025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:56.292025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:56.292025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:56.292025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:56.293019 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:56.293019 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:56.293019 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:56.293019 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:56.293019 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:57:56.293019 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:57:56.293019 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:57:56.293019 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:57:56.293019 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:57:56.293019 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:57:56.293019 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:57:56.293019 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:57:56.294817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 124/239] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:57:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:57:56.300570 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1afb65-5afb-49ad-a7cf-5577776c1a1e None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:56.351406 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7aea40-efd0-4a9f-a619-0bea87efd326 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:56.371850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:56.401704 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e41f78-e7d9-4c3e-89bb-8a9115a4103a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:56.402572 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e41f78-e7d9-4c3e-89bb-8a9115a4103a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:56.402929 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e41f78-e7d9-4c3e-89bb-8a9115a4103a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:56.403207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e41f78-e7d9-4c3e-89bb-8a9115a4103a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:56.405255 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e41f78-e7d9-4c3e-89bb-8a9115a4103a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:56.406752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e41f78-e7d9-4c3e-89bb-8a9115a4103a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:56.408926 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 116/240] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:57:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:56.415529 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e41f78-e7d9-4c3e-89bb-8a9115a4103a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:56.433464 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1afb65-5afb-49ad-a7cf-5577776c1a1e None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:56.452238 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:56.470902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91dc8d-7b00-491d-aaf6-541e04dd0e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:56.471676 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91dc8d-7b00-491d-aaf6-541e04dd0e8d None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:56.472035 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91dc8d-7b00-491d-aaf6-541e04dd0e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:56.472409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91dc8d-7b00-491d-aaf6-541e04dd0e8d None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:56.473238 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91dc8d-7b00-491d-aaf6-541e04dd0e8d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:56.473618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91dc8d-7b00-491d-aaf6-541e04dd0e8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:56.475134 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91dc8d-7b00-491d-aaf6-541e04dd0e8d None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:56.484615 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 125/241] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 15:57:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:57:56.490279 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91dc8d-7b00-491d-aaf6-541e04dd0e8d None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:56.597160 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91dc8d-7b00-491d-aaf6-541e04dd0e8d None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:56.600657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e41f78-e7d9-4c3e-89bb-8a9115a4103a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:56.640113 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878bdbf-64e1-4d37-a9c5-5a4c9c2539bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:56.640919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878bdbf-64e1-4d37-a9c5-5a4c9c2539bb None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:56.641206 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878bdbf-64e1-4d37-a9c5-5a4c9c2539bb None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:56.641491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878bdbf-64e1-4d37-a9c5-5a4c9c2539bb None admin] PATH_INFO: `/v3/users/1c7eec7c2122493d9c841b67b3c9d281` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:56.671138 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1878bdbf-64e1-4d37-a9c5-5a4c9c2539bb None admin] RBAC: Authorizing `identity:get_user(user_id=1c7eec7c2122493d9c841b67b3c9d281)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:56.674445 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1878bdbf-64e1-4d37-a9c5-5a4c9c2539bb None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:56.677823 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f48bb8-e4c0-42cf-82d3-6af1a13b47db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:56.678526 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f48bb8-e4c0-42cf-82d3-6af1a13b47db None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:56.678811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f48bb8-e4c0-42cf-82d3-6af1a13b47db None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:56.679089 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f48bb8-e4c0-42cf-82d3-6af1a13b47db None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:56.679809 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f48bb8-e4c0-42cf-82d3-6af1a13b47db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:56.681254 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f48bb8-e4c0-42cf-82d3-6af1a13b47db None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:56.686291 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 126/242] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:57:56 2022] GET /identity/v3/users/1c7eec7c2122493d9c841b67b3c9d281 => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:56.694158 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878bdbf-64e1-4d37-a9c5-5a4c9c2539bb None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:56.724983 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 117/243] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 15:57:56 2022] POST /identity/v3/users => generated 293 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:57:56.766400 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:56.828520 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878bdbf-64e1-4d37-a9c5-5a4c9c2539bb None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:56.868569 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34772b9-69fd-4df7-ab18-da2ff12be888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:56.869305 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34772b9-69fd-4df7-ab18-da2ff12be888 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:56.869598 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34772b9-69fd-4df7-ab18-da2ff12be888 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:56.869877 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34772b9-69fd-4df7-ab18-da2ff12be888 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:56.951294 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34772b9-69fd-4df7-ab18-da2ff12be888 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:56.952749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34772b9-69fd-4df7-ab18-da2ff12be888 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:56.990248 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb62aa-e83a-4a0f-8535-8d423f7bea23 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:56.990759 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb62aa-e83a-4a0f-8535-8d423f7bea23 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:56.990833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb62aa-e83a-4a0f-8535-8d423f7bea23 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:56.992516 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 118/244] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:57:56.995919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:56.999475 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafc00c-e4d9-4c00-981c-f625180cee64 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:57.000021 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafc00c-e4d9-4c00-981c-f625180cee64 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:57.000505 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafc00c-e4d9-4c00-981c-f625180cee64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:57.023284 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-fdafc00c-e4d9-4c00-981c-f625180cee64 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:57.024920 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-fdafc00c-e4d9-4c00-981c-f625180cee64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:57.065578 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:57.072685 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 127/245] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:57:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 15:57:57.079191 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34772b9-69fd-4df7-ab18-da2ff12be888 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:57.103605 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 119/246] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:57.128903 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdafc00c-e4d9-4c00-981c-f625180cee64 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:57.183854 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34772b9-69fd-4df7-ab18-da2ff12be888 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:57.223012 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf56d1-89e5-4cb2-b4d7-584b471f23e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:57.223824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf56d1-89e5-4cb2-b4d7-584b471f23e2 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:57.224112 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf56d1-89e5-4cb2-b4d7-584b471f23e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:57.224395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf56d1-89e5-4cb2-b4d7-584b471f23e2 None admin] PATH_INFO: `/v3/projects/328b693aa62e4467aa34a749006ad00f` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:57.227921 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcf56d1-89e5-4cb2-b4d7-584b471f23e2 None admin] RBAC: Authorizing `identity:get_project(project_id=328b693aa62e4467aa34a749006ad00f)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:57.227921 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcf56d1-89e5-4cb2-b4d7-584b471f23e2 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:57.229578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 128/247] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:57:57 2022] GET /identity/v3/projects/328b693aa62e4467aa34a749006ad00f => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:57.236852 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf56d1-89e5-4cb2-b4d7-584b471f23e2 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:57.249134 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:57.254442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdafc00c-e4d9-4c00-981c-f625180cee64 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:57.304552 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e7d342-1dc5-417a-81db-ca02ff91a98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:57.305861 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7d342-1dc5-417a-81db-ca02ff91a98a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:57.306377 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7d342-1dc5-417a-81db-ca02ff91a98a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:57.306856 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7d342-1dc5-417a-81db-ca02ff91a98a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:57.321486 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e7d342-1dc5-417a-81db-ca02ff91a98a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:57:57.331449 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-c1e7d342-1dc5-417a-81db-ca02ff91a98a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:57:57.331449 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:57.331449 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:57.331449 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:57.331449 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:57.331449 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:57.331449 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:57.331449 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:57.331449 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:57.331449 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:57.331449 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:57.331449 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:57.331449 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:57:57.331449 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:57:57.331449 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:57:57.331449 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:57:57.332349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:57:57.332349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:57:57.332349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:57:57.332349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:57:57.332349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:57.332349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:57.332349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:57.332349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:57.332349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:57.332349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:57.332349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:57.332349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:57.332349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:57.332349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:57.332349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:57.332349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:57.333141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:57.333141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:57.333141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:57.333141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:57.333141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:57:57.333141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:57:57.333141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:57:57.333141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:57:57.333141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:57:57.333141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:57:57.333141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:57:57.333141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:57:57.335974 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 120/248] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:57:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:57:57.342529 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e7d342-1dc5-417a-81db-ca02ff91a98a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:57.351425 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf56d1-89e5-4cb2-b4d7-584b471f23e2 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:57.379990 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:57.391286 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b6b671-b8fc-487f-b392-8a6ded09e411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:57.391957 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6b671-b8fc-487f-b392-8a6ded09e411 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:57.392246 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6b671-b8fc-487f-b392-8a6ded09e411 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:57.392527 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6b671-b8fc-487f-b392-8a6ded09e411 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:57.393375 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b6b671-b8fc-487f-b392-8a6ded09e411 None admin] RBAC: Adding query filter params (role.id=1a524a543caa49b286168391bb19d71e, scope.project.id=328b693aa62e4467aa34a749006ad00f, user.id=1c7eec7c2122493d9c841b67b3c9d281) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:57.393692 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b6b671-b8fc-487f-b392-8a6ded09e411 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:57.395211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b6b671-b8fc-487f-b392-8a6ded09e411 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:57.402278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 129/249] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:57:57 2022] GET /identity/v3/role_assignments?user.id=1c7eec7c2122493d9c841b67b3c9d281&scope.project.id=328b693aa62e4467aa34a749006ad00f&role.id=1a524a543caa49b286168391bb19d71e => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:57.469367 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e7d342-1dc5-417a-81db-ca02ff91a98a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:57.511415 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50bfd17-72ab-40b6-8153-14b70acf9cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:57.512510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bfd17-72ab-40b6-8153-14b70acf9cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:57.513072 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bfd17-72ab-40b6-8153-14b70acf9cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:57.513553 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bfd17-72ab-40b6-8153-14b70acf9cb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:57.514692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50bfd17-72ab-40b6-8153-14b70acf9cb9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:57.515252 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50bfd17-72ab-40b6-8153-14b70acf9cb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:57.516891 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50bfd17-72ab-40b6-8153-14b70acf9cb9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:57.527336 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 121/250] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 15:57:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:57:57.533683 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b6b671-b8fc-487f-b392-8a6ded09e411 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:57.627169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:57.663474 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b6b671-b8fc-487f-b392-8a6ded09e411 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:57.704036 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dea9aa-237b-46cd-8ab2-69dbe4444960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:57.705321 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dea9aa-237b-46cd-8ab2-69dbe4444960 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:57.705806 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dea9aa-237b-46cd-8ab2-69dbe4444960 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:57.706324 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dea9aa-237b-46cd-8ab2-69dbe4444960 None admin] PATH_INFO: `/v3/users/3cbb634531974782aee895d70907f60c` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:57.707819 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dea9aa-237b-46cd-8ab2-69dbe4444960 None admin] RBAC: Authorizing `identity:get_user(user_id=3cbb634531974782aee895d70907f60c)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:57.709591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dea9aa-237b-46cd-8ab2-69dbe4444960 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:57.711517 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 130/251] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:57:57 2022] GET /identity/v3/users/3cbb634531974782aee895d70907f60c => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:57.717346 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50bfd17-72ab-40b6-8153-14b70acf9cb9 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:57.840112 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:57.849085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50bfd17-72ab-40b6-8153-14b70acf9cb9 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:57.896610 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1e690-9420-4117-a9ec-9f116fb39046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:57.897842 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1e690-9420-4117-a9ec-9f116fb39046 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:57.898352 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1e690-9420-4117-a9ec-9f116fb39046 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:57.898827 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1e690-9420-4117-a9ec-9f116fb39046 None admin] PATH_INFO: `/v3/projects/328b693aa62e4467aa34a749006ad00f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:57.900374 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1e690-9420-4117-a9ec-9f116fb39046 None admin] RBAC: Authorizing `identity:get_project(project_id=328b693aa62e4467aa34a749006ad00f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:57.902115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1e690-9420-4117-a9ec-9f116fb39046 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:57.904645 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 122/252] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:57:57 2022] GET /identity/v3/projects/328b693aa62e4467aa34a749006ad00f => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:57.910649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dea9aa-237b-46cd-8ab2-69dbe4444960 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:58.045902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dea9aa-237b-46cd-8ab2-69dbe4444960 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:58.064857 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49142f-fedb-421c-8053-57c1059f4dee None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:58.065808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49142f-fedb-421c-8053-57c1059f4dee None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:58.066348 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49142f-fedb-421c-8053-57c1059f4dee None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:58.068371 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 123/253] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:58.075478 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659c48d-a231-48f5-a4ab-877f34139280 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:58.076088 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659c48d-a231-48f5-a4ab-877f34139280 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:58.076569 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659c48d-a231-48f5-a4ab-877f34139280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:58.101952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e6ca08-5489-44b4-b1aa-a2a1ebf4b8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:58.101952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6ca08-5489-44b4-b1aa-a2a1ebf4b8a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:58.101952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6ca08-5489-44b4-b1aa-a2a1ebf4b8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:58.101952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6ca08-5489-44b4-b1aa-a2a1ebf4b8a3 None admin] PATH_INFO: `/v3/projects/328b693aa62e4467aa34a749006ad00f/users/3cbb634531974782aee895d70907f60c/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:58.102356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e6ca08-5489-44b4-b1aa-a2a1ebf4b8a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=328b693aa62e4467aa34a749006ad00f, user_id=3cbb634531974782aee895d70907f60c, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:58.102356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e6ca08-5489-44b4-b1aa-a2a1ebf4b8a3 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:58.115085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f659c48d-a231-48f5-a4ab-877f34139280 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:58.120864 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:58.122481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f659c48d-a231-48f5-a4ab-877f34139280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:58.127110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 131/254] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 15:57:57 2022] PUT /identity/v3/projects/328b693aa62e4467aa34a749006ad00f/users/3cbb634531974782aee895d70907f60c/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:57:58.228763 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 124/255] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:57:58.277658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e6ca08-5489-44b4-b1aa-a2a1ebf4b8a3 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:58.420255 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e6ca08-5489-44b4-b1aa-a2a1ebf4b8a3 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:58.465793 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735d5c5-8d44-42f2-98d2-9a7741370d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:58.467861 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735d5c5-8d44-42f2-98d2-9a7741370d0d None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:58.468474 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735d5c5-8d44-42f2-98d2-9a7741370d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:58.469026 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735d5c5-8d44-42f2-98d2-9a7741370d0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:58.488474 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0735d5c5-8d44-42f2-98d2-9a7741370d0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:57:58.496964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-0735d5c5-8d44-42f2-98d2-9a7741370d0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:57:58.496964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:58.496964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:58.496964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:58.496964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:58.496964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:58.496964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:58.496964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:58.496964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:58.496964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:58.496964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:58.496964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:58.496964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:57:58.496964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:57:58.496964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:57:58.496964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:57:58.502870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:57:58.502870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:57:58.502870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:57:58.502870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:57:58.502870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:58.502870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:58.502870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:58.502870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:58.502870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:58.502870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:58.502870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:58.502870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:58.502870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:58.502870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:58.502870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:58.502870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:58.503689 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:58.503689 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:58.503689 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:58.503689 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:58.503689 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:57:58.503689 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:57:58.503689 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:57:58.503689 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:57:58.503689 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:57:58.503689 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:57:58.503689 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:57:58.503689 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:57:58.506897 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 132/256] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 15:57:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:57:58.512690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659c48d-a231-48f5-a4ab-877f34139280 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:58.737144 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:58.760671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659c48d-a231-48f5-a4ab-877f34139280 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:58.767887 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972e4d9-5168-4ce1-84b4-c1f1dda85e54 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:58.768505 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972e4d9-5168-4ce1-84b4-c1f1dda85e54 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:58.769003 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972e4d9-5168-4ce1-84b4-c1f1dda85e54 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:58.770882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 133/257] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:57:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:57:58.788224 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81010cc9-87d9-4e8f-b8de-ecc4ffc8479b None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:58.788833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81010cc9-87d9-4e8f-b8de-ecc4ffc8479b None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:58.789437 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81010cc9-87d9-4e8f-b8de-ecc4ffc8479b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:58.820311 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0304ded-9003-40ef-983c-e3dfcc107308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:58.821314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0304ded-9003-40ef-983c-e3dfcc107308 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:58.821802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0304ded-9003-40ef-983c-e3dfcc107308 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:58.822275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0304ded-9003-40ef-983c-e3dfcc107308 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:58.823217 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0304ded-9003-40ef-983c-e3dfcc107308 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:58.823704 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0304ded-9003-40ef-983c-e3dfcc107308 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:58.825464 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0304ded-9003-40ef-983c-e3dfcc107308 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:58.836222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 125/258] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:57:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:58.837757 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-81010cc9-87d9-4e8f-b8de-ecc4ffc8479b None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:57:58.841706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-81010cc9-87d9-4e8f-b8de-ecc4ffc8479b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:57:58.843438 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0304ded-9003-40ef-983c-e3dfcc107308 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:58.926272 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:58.956608 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 134/259] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:57:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:57:58.985607 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81010cc9-87d9-4e8f-b8de-ecc4ffc8479b None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:59.032400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0304ded-9003-40ef-983c-e3dfcc107308 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:59.074140 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb53523-17bd-4cf1-b8b1-914393c096a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:59.076482 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb53523-17bd-4cf1-b8b1-914393c096a3 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:59.076821 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb53523-17bd-4cf1-b8b1-914393c096a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:59.078238 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb53523-17bd-4cf1-b8b1-914393c096a3 None admin] PATH_INFO: `/v3/users/71e063c2f41c4034a33858116dcb702f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:59.079803 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb53523-17bd-4cf1-b8b1-914393c096a3 None admin] RBAC: Authorizing `identity:get_user(user_id=71e063c2f41c4034a33858116dcb702f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:59.081295 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb53523-17bd-4cf1-b8b1-914393c096a3 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:59.090280 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 126/260] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:57:58 2022] GET /identity/v3/users/71e063c2f41c4034a33858116dcb702f => generated 283 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:59.095930 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb53523-17bd-4cf1-b8b1-914393c096a3 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:59.130450 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81010cc9-87d9-4e8f-b8de-ecc4ffc8479b None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:59.141694 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:59.202547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886f0b7-e45c-470b-bdd9-42b9d5b15e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:59.203824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886f0b7-e45c-470b-bdd9-42b9d5b15e07 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:59.204977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886f0b7-e45c-470b-bdd9-42b9d5b15e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:59.205496 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886f0b7-e45c-470b-bdd9-42b9d5b15e07 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:59.225632 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886f0b7-e45c-470b-bdd9-42b9d5b15e07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:57:59.238754 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-2886f0b7-e45c-470b-bdd9-42b9d5b15e07 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:57:59.238754 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:57:59.238754 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:57:59.238754 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:57:59.238754 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:57:59.238754 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:57:59.238754 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:57:59.238754 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:57:59.238754 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:57:59.238754 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:57:59.238754 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:57:59.238754 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:57:59.238754 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:57:59.238754 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:57:59.238754 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:57:59.238754 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:57:59.239716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:57:59.239716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:57:59.239716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:57:59.239716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:57:59.239716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:57:59.239716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:57:59.239716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:57:59.239716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:57:59.239716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:57:59.239716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:57:59.239716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:57:59.239716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:57:59.239716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:57:59.239716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:57:59.239716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:57:59.239716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:57:59.240507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:57:59.240507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:57:59.240507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:57:59.240507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:57:59.240507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:57:59.240507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:57:59.240507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:57:59.240507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:57:59.240507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:57:59.240507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:57:59.240507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:57:59.240507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:57:59.243517 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 135/261] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 15:57:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:57:59.254774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886f0b7-e45c-470b-bdd9-42b9d5b15e07 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:59.351425 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb53523-17bd-4cf1-b8b1-914393c096a3 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:59.365033 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:59.421273 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f15f9-5e25-4bed-bb1d-590e2d472357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:59.422666 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f15f9-5e25-4bed-bb1d-590e2d472357 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:59.423593 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f15f9-5e25-4bed-bb1d-590e2d472357 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:59.424605 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f15f9-5e25-4bed-bb1d-590e2d472357 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:59.463941 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886f0b7-e45c-470b-bdd9-42b9d5b15e07 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:59.521578 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f15f9-5e25-4bed-bb1d-590e2d472357 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:59.523489 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f15f9-5e25-4bed-bb1d-590e2d472357 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:59.551213 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:59.558483 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfda4e4-1f63-4a72-a7e2-5cd5b3195e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:59.559175 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfda4e4-1f63-4a72-a7e2-5cd5b3195e32 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:59.559464 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfda4e4-1f63-4a72-a7e2-5cd5b3195e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:59.559833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfda4e4-1f63-4a72-a7e2-5cd5b3195e32 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:59.560813 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfda4e4-1f63-4a72-a7e2-5cd5b3195e32 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:57:59.561136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfda4e4-1f63-4a72-a7e2-5cd5b3195e32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:59.562820 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfda4e4-1f63-4a72-a7e2-5cd5b3195e32 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:59.588953 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 136/262] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:57:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:59.595073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfda4e4-1f63-4a72-a7e2-5cd5b3195e32 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:59.641964 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 127/263] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:57:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 15:57:59.651255 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f15f9-5e25-4bed-bb1d-590e2d472357 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:59.773487 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f15f9-5e25-4bed-bb1d-590e2d472357 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:59.822120 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfda4e4-1f63-4a72-a7e2-5cd5b3195e32 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:57:59.838312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:59.854450 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:57:59.864286 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8a0f8-324d-48e2-859d-7bec2c0ecf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:59.865838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8a0f8-324d-48e2-859d-7bec2c0ecf32 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:59.866398 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8a0f8-324d-48e2-859d-7bec2c0ecf32 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:59.866883 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8a0f8-324d-48e2-859d-7bec2c0ecf32 None admin] PATH_INFO: `/v3/projects/328b693aa62e4467aa34a749006ad00f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:59.871810 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c8a0f8-324d-48e2-859d-7bec2c0ecf32 None admin] RBAC: Authorizing `identity:get_project(project_id=328b693aa62e4467aa34a749006ad00f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:59.874771 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c8a0f8-324d-48e2-859d-7bec2c0ecf32 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:59.886716 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 128/264] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:57:59 2022] GET /identity/v3/projects/328b693aa62e4467aa34a749006ad00f => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:59.888693 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8a0f8-324d-48e2-859d-7bec2c0ecf32 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:57:59.912082 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad550f23-7c2d-4d45-9d44-d3416d4172d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:57:59.913290 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad550f23-7c2d-4d45-9d44-d3416d4172d9 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:57:59.913753 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad550f23-7c2d-4d45-9d44-d3416d4172d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:57:59.914226 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad550f23-7c2d-4d45-9d44-d3416d4172d9 None admin] PATH_INFO: `/v3/users/71e063c2f41c4034a33858116dcb702f` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:57:59.916142 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad550f23-7c2d-4d45-9d44-d3416d4172d9 None admin] RBAC: Authorizing `identity:get_user(user_id=71e063c2f41c4034a33858116dcb702f)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:57:59.918024 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad550f23-7c2d-4d45-9d44-d3416d4172d9 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:57:59.920092 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 137/265] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:57:59 2022] GET /identity/v3/users/71e063c2f41c4034a33858116dcb702f => generated 283 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:57:59.927828 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad550f23-7c2d-4d45-9d44-d3416d4172d9 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:00.089790 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8a0f8-324d-48e2-859d-7bec2c0ecf32 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:00.141092 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad550f23-7c2d-4d45-9d44-d3416d4172d9 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:00.152173 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397cf47-b1e8-4e35-b7e8-fde1f4ec6b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:00.152888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397cf47-b1e8-4e35-b7e8-fde1f4ec6b7d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:00.153197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397cf47-b1e8-4e35-b7e8-fde1f4ec6b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:00.153562 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397cf47-b1e8-4e35-b7e8-fde1f4ec6b7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:00.154729 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397cf47-b1e8-4e35-b7e8-fde1f4ec6b7d None admin] RBAC: Adding query filter params (role.id=bbbc77ec645d4ff5a8bf29ce0377f578, scope.project.id=328b693aa62e4467aa34a749006ad00f, user.id=71e063c2f41c4034a33858116dcb702f) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:00.155073 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397cf47-b1e8-4e35-b7e8-fde1f4ec6b7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:00.156651 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397cf47-b1e8-4e35-b7e8-fde1f4ec6b7d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:00.168135 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 129/266] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:57:59 2022] GET /identity/v3/role_assignments?user.id=71e063c2f41c4034a33858116dcb702f&scope.project.id=328b693aa62e4467aa34a749006ad00f&role.id=bbbc77ec645d4ff5a8bf29ce0377f578 => generated 608 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:00.204511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ceefa-8dc3-42ac-bc2a-5ec4624c9e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:00.205193 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ceefa-8dc3-42ac-bc2a-5ec4624c9e6f None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:00.205482 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ceefa-8dc3-42ac-bc2a-5ec4624c9e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:00.205762 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ceefa-8dc3-42ac-bc2a-5ec4624c9e6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:00.277778 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:00.318940 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382ceefa-8dc3-42ac-bc2a-5ec4624c9e6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:00.320478 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382ceefa-8dc3-42ac-bc2a-5ec4624c9e6f None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:00.456599 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 138/267] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:57:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:00.462378 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397cf47-b1e8-4e35-b7e8-fde1f4ec6b7d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:00.576304 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:00.646683 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397cf47-b1e8-4e35-b7e8-fde1f4ec6b7d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:00.695481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad6fc1-3600-47d7-9f5c-bb48fbadbfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:00.696379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad6fc1-3600-47d7-9f5c-bb48fbadbfac None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:00.696664 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad6fc1-3600-47d7-9f5c-bb48fbadbfac None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:00.696936 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad6fc1-3600-47d7-9f5c-bb48fbadbfac None admin] PATH_INFO: `/v3/projects/328b693aa62e4467aa34a749006ad00f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:00.702922 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad6fc1-3600-47d7-9f5c-bb48fbadbfac None admin] RBAC: Authorizing `identity:get_project(project_id=328b693aa62e4467aa34a749006ad00f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:00.704445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad6fc1-3600-47d7-9f5c-bb48fbadbfac None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:00.706744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 130/268] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:00 2022] GET /identity/v3/projects/328b693aa62e4467aa34a749006ad00f => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:00.712827 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ceefa-8dc3-42ac-bc2a-5ec4624c9e6f None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:00.819498 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:00.847630 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ceefa-8dc3-42ac-bc2a-5ec4624c9e6f None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:00.865730 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf733e-23d7-4ee1-9232-c74831120a6d None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:00.866078 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf733e-23d7-4ee1-9232-c74831120a6d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:00.866327 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf733e-23d7-4ee1-9232-c74831120a6d None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:00.868008 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 131/269] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:00.874491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f624ea-016a-41b7-8753-0b9449b7e08d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:00.875119 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f624ea-016a-41b7-8753-0b9449b7e08d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:00.875601 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f624ea-016a-41b7-8753-0b9449b7e08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:00.893196 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020cb4c-328e-4669-9bd8-e4bbe661b5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:00.893860 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020cb4c-328e-4669-9bd8-e4bbe661b5cb None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:00.894177 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020cb4c-328e-4669-9bd8-e4bbe661b5cb None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:00.894454 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020cb4c-328e-4669-9bd8-e4bbe661b5cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:00.895429 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020cb4c-328e-4669-9bd8-e4bbe661b5cb None admin] RBAC: Adding query filter params (role.id=e0b0c216ef51441ab34b6e036da09daa, scope.project.id=328b693aa62e4467aa34a749006ad00f, user.id=71e063c2f41c4034a33858116dcb702f) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:00.895754 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020cb4c-328e-4669-9bd8-e4bbe661b5cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:00.897393 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020cb4c-328e-4669-9bd8-e4bbe661b5cb None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:00.905148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 139/270] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:58:00 2022] GET /identity/v3/role_assignments?user.id=71e063c2f41c4034a33858116dcb702f&scope.project.id=328b693aa62e4467aa34a749006ad00f&role.id=e0b0c216ef51441ab34b6e036da09daa => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:00.920224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-52f624ea-016a-41b7-8753-0b9449b7e08d None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:00.922301 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-52f624ea-016a-41b7-8753-0b9449b7e08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:01.043195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 132/271] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:01.082944 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020cb4c-328e-4669-9bd8-e4bbe661b5cb None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:01.176996 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf41841-65ea-4226-b09e-605c1a289724 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:01.177974 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf41841-65ea-4226-b09e-605c1a289724 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:01.178476 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf41841-65ea-4226-b09e-605c1a289724 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:01.180200 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 133/272] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:58:01.186940 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894916b-8c38-4220-a4d7-eb83379a8dec None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:01.187637 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894916b-8c38-4220-a4d7-eb83379a8dec None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:01.188123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894916b-8c38-4220-a4d7-eb83379a8dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:01.213737 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d894916b-8c38-4220-a4d7-eb83379a8dec None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:01.215382 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d894916b-8c38-4220-a4d7-eb83379a8dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:01.228885 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020cb4c-328e-4669-9bd8-e4bbe661b5cb None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:01.271546 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56deea8-f346-4ca7-8a28-39a59eb44048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:01.272371 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56deea8-f346-4ca7-8a28-39a59eb44048 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:01.272651 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56deea8-f346-4ca7-8a28-39a59eb44048 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:01.272928 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56deea8-f346-4ca7-8a28-39a59eb44048 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:01.274495 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56deea8-f346-4ca7-8a28-39a59eb44048 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:01.276309 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56deea8-f346-4ca7-8a28-39a59eb44048 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:01.278492 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 140/273] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:01.280349 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b55a0-93ef-48eb-b1be-7abf9bf52b69 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:01.280643 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b55a0-93ef-48eb-b1be-7abf9bf52b69 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:01.280920 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b55a0-93ef-48eb-b1be-7abf9bf52b69 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:01.282404 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 141/274] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:01.283090 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:01.289132 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59d2d9-24f1-4766-b2bc-674db9304de1 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:01.289508 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59d2d9-24f1-4766-b2bc-674db9304de1 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:01.289983 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59d2d9-24f1-4766-b2bc-674db9304de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:01.312199 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-bf59d2d9-24f1-4766-b2bc-674db9304de1 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:01.313423 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-bf59d2d9-24f1-4766-b2bc-674db9304de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:01.317702 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:01.320009 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 134/275] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:01.321385 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894916b-8c38-4220-a4d7-eb83379a8dec None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:01.405948 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 142/276] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:01.405948 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf59d2d9-24f1-4766-b2bc-674db9304de1 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:01.453660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894916b-8c38-4220-a4d7-eb83379a8dec None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:01.490929 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516175a4-c364-4d89-adb2-a16b5743f5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:01.491500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516175a4-c364-4d89-adb2-a16b5743f5b2 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:01.491755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516175a4-c364-4d89-adb2-a16b5743f5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:01.492027 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516175a4-c364-4d89-adb2-a16b5743f5b2 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:01.492729 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516175a4-c364-4d89-adb2-a16b5743f5b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:01.494364 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516175a4-c364-4d89-adb2-a16b5743f5b2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:01.515971 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf59d2d9-24f1-4766-b2bc-674db9304de1 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:01.525462 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 135/277] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 15:58:01 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:58:01.525462 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516175a4-c364-4d89-adb2-a16b5743f5b2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:01.560966 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09077c86-c532-4345-bc60-8c8faacc55ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:01.561789 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09077c86-c532-4345-bc60-8c8faacc55ca None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:01.562126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09077c86-c532-4345-bc60-8c8faacc55ca None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:01.562459 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09077c86-c532-4345-bc60-8c8faacc55ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:01.571909 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:01.575723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09077c86-c532-4345-bc60-8c8faacc55ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:01.585839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-09077c86-c532-4345-bc60-8c8faacc55ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:01.585839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:01.585839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:01.585839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:01.585839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:01.585839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:01.585839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:01.585839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:01.585839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:01.585839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:01.585839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:01.585839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:01.585839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:01.585839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:01.585839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:01.585839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:01.586867 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:01.586867 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:01.586867 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:01.586867 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:01.586867 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:01.586867 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:01.586867 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:01.586867 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:01.586867 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:01.586867 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:01.586867 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:01.586867 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:01.586867 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:01.586867 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:01.586867 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:01.586867 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:01.587658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:01.587658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:01.587658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:01.587658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:01.587658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:01.587658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:01.587658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:01.587658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:01.587658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:01.587658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:01.587658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:01.587658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:01.588462 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 143/278] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:01.594711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:01.596059 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09077c86-c532-4345-bc60-8c8faacc55ca None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:01.681007 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516175a4-c364-4d89-adb2-a16b5743f5b2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:01.722369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07271c-5c17-4f73-92c7-e786d9b9286d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:01.723515 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07271c-5c17-4f73-92c7-e786d9b9286d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:01.723826 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07271c-5c17-4f73-92c7-e786d9b9286d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:01.724069 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07271c-5c17-4f73-92c7-e786d9b9286d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:01.739582 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd07271c-5c17-4f73-92c7-e786d9b9286d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:01.748983 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-dd07271c-5c17-4f73-92c7-e786d9b9286d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:58:01.748983 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:01.748983 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:01.748983 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:01.748983 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:01.748983 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:01.748983 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:01.748983 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:01.748983 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:01.748983 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:01.748983 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:01.748983 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:01.748983 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:01.748983 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:01.748983 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:01.748983 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:01.750188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:01.750188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:01.750188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:01.750188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:01.750188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:01.750188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:01.750188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:01.750188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:01.750188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:01.750188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:01.750188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:01.750188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:01.750188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:01.750188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:01.750188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:01.750188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:01.751178 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:01.751178 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:01.751178 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:01.751178 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:01.751178 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:01.751178 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:01.751178 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:01.751178 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:01.751178 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:01.751178 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:01.751178 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:58:01.751178 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:01.753314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 136/279] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:01.754174 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09077c86-c532-4345-bc60-8c8faacc55ca None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:01.759882 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07271c-5c17-4f73-92c7-e786d9b9286d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:01.805201 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3d69ea-839e-47bc-b23b-8b442a45a808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:01.805881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d69ea-839e-47bc-b23b-8b442a45a808 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:01.806198 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d69ea-839e-47bc-b23b-8b442a45a808 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:01.806478 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d69ea-839e-47bc-b23b-8b442a45a808 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:01.807251 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3d69ea-839e-47bc-b23b-8b442a45a808 None admin] RBAC: Adding query filter params (name=member) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:01.807828 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3d69ea-839e-47bc-b23b-8b442a45a808 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:01.808980 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3d69ea-839e-47bc-b23b-8b442a45a808 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:01.818820 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 144/280] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 15:58:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:01.820100 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e94239-6cee-421c-8d69-66de51df6fe7 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:01.820388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e94239-6cee-421c-8d69-66de51df6fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:01.820668 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e94239-6cee-421c-8d69-66de51df6fe7 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:01.822107 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 145/281] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:01.829553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e21cf-06ce-4426-b66d-c04199e4e043 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:01.829845 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e21cf-06ce-4426-b66d-c04199e4e043 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:01.830152 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e21cf-06ce-4426-b66d-c04199e4e043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:01.850518 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-8c6e21cf-06ce-4426-b66d-c04199e4e043 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:01.852004 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-8c6e21cf-06ce-4426-b66d-c04199e4e043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:01.897772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:01.902563 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07271c-5c17-4f73-92c7-e786d9b9286d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:01.928845 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:01.942478 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 146/282] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:01.943417 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e21cf-06ce-4426-b66d-c04199e4e043 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:01.944953 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52ecfb-be42-4373-a1b8-bd0098d13a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:01.945641 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52ecfb-be42-4373-a1b8-bd0098d13a98 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:01.945940 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52ecfb-be42-4373-a1b8-bd0098d13a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:01.946233 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52ecfb-be42-4373-a1b8-bd0098d13a98 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:01.946988 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52ecfb-be42-4373-a1b8-bd0098d13a98 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:01.947299 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52ecfb-be42-4373-a1b8-bd0098d13a98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:01.948685 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52ecfb-be42-4373-a1b8-bd0098d13a98 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:01.962868 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 137/283] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 15:58:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:01.964220 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52ecfb-be42-4373-a1b8-bd0098d13a98 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:02.058561 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e21cf-06ce-4426-b66d-c04199e4e043 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:02.087930 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52ecfb-be42-4373-a1b8-bd0098d13a98 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:02.094634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4c714-2f24-468f-9f3b-7dfafc58764b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:02.095517 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4c714-2f24-468f-9f3b-7dfafc58764b None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:02.095802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4c714-2f24-468f-9f3b-7dfafc58764b None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:02.096076 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4c714-2f24-468f-9f3b-7dfafc58764b None admin] PATH_INFO: `/v3/users/80f3e90f5e5746f7bbf9156097c21d39` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:02.111290 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf4c714-2f24-468f-9f3b-7dfafc58764b None admin] RBAC: Authorizing `identity:get_user(user_id=80f3e90f5e5746f7bbf9156097c21d39)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:02.112835 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf4c714-2f24-468f-9f3b-7dfafc58764b None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:02.114784 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 147/284] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:01 2022] GET /identity/v3/users/80f3e90f5e5746f7bbf9156097c21d39 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:02.115463 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4c714-2f24-468f-9f3b-7dfafc58764b None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:02.130236 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e19e7-0570-4f66-88e6-60a8da37c19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:02.131133 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e19e7-0570-4f66-88e6-60a8da37c19a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:02.131676 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e19e7-0570-4f66-88e6-60a8da37c19a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:02.132060 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e19e7-0570-4f66-88e6-60a8da37c19a None admin] PATH_INFO: `/v3/users/1c7eec7c2122493d9c841b67b3c9d281` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:02.133470 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e19e7-0570-4f66-88e6-60a8da37c19a None admin] RBAC: Authorizing `identity:get_user(user_id=1c7eec7c2122493d9c841b67b3c9d281)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:02.134973 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e19e7-0570-4f66-88e6-60a8da37c19a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:02.137186 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 138/285] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:01 2022] GET /identity/v3/users/1c7eec7c2122493d9c841b67b3c9d281 => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:02.138033 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e19e7-0570-4f66-88e6-60a8da37c19a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:02.170630 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:02.175057 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:02.232176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4c714-2f24-468f-9f3b-7dfafc58764b None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:02.269268 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e19e7-0570-4f66-88e6-60a8da37c19a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:02.271742 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d8123-db90-4fb7-90e7-c8ca71f6122e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:02.272566 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d8123-db90-4fb7-90e7-c8ca71f6122e None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:02.272996 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d8123-db90-4fb7-90e7-c8ca71f6122e None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:02.273277 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d8123-db90-4fb7-90e7-c8ca71f6122e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:02.285397 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d8123-db90-4fb7-90e7-c8ca71f6122e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:02.294629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-154d8123-db90-4fb7-90e7-c8ca71f6122e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:02.294629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:02.294629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:02.294629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:02.294629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:02.294629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:02.294629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:02.294629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:02.294629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:02.294629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:02.294629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:02.294629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:02.294629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:02.294629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:02.294629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:02.294629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:02.295524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:02.295524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:02.295524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:02.295524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:02.295524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:02.295524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:02.295524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:02.295524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:02.295524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:02.295524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:02.295524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:02.295524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:02.295524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:02.295524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:02.295524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:02.295524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:02.296312 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:02.296312 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:02.296312 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:02.296312 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:02.296312 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:02.296312 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:02.296312 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:02.296312 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:02.296312 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:02.296312 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:02.296312 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:02.296312 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:02.296910 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 148/286] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:02.301957 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d8123-db90-4fb7-90e7-c8ca71f6122e None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:02.311108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb552035-68d5-431d-b76f-7ad385c6f2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:02.311984 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb552035-68d5-431d-b76f-7ad385c6f2c8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:02.312696 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb552035-68d5-431d-b76f-7ad385c6f2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:02.312983 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb552035-68d5-431d-b76f-7ad385c6f2c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:02.314164 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb552035-68d5-431d-b76f-7ad385c6f2c8 None admin] RBAC: Adding query filter params (role.id=3ce570490865493380e6941d004136bd, user.id=80f3e90f5e5746f7bbf9156097c21d39, scope.system=all) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:02.314486 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb552035-68d5-431d-b76f-7ad385c6f2c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:02.316698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb552035-68d5-431d-b76f-7ad385c6f2c8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:02.323834 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 139/287] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jun 16 15:58:02 2022] GET /identity/v3/role_assignments?user.id=80f3e90f5e5746f7bbf9156097c21d39&scope.system=all&role.id=3ce570490865493380e6941d004136bd => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:02.326488 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb552035-68d5-431d-b76f-7ad385c6f2c8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:02.474187 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:02.487044 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb552035-68d5-431d-b76f-7ad385c6f2c8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:02.533241 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74be96-4186-472c-8d14-7bc797413fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:02.533958 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74be96-4186-472c-8d14-7bc797413fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:02.534260 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74be96-4186-472c-8d14-7bc797413fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:02.534541 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74be96-4186-472c-8d14-7bc797413fb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:02.535326 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f74be96-4186-472c-8d14-7bc797413fb6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:02.535644 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f74be96-4186-472c-8d14-7bc797413fb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:02.537074 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d8123-db90-4fb7-90e7-c8ca71f6122e None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:02.537295 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f74be96-4186-472c-8d14-7bc797413fb6 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:02.549065 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 140/288] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 15:58:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:02.562473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74be96-4186-472c-8d14-7bc797413fb6 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:02.567877 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:02.599549 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd261c0-5018-4e0b-995e-f93129a1ff1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:02.600802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd261c0-5018-4e0b-995e-f93129a1ff1d None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:02.601090 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd261c0-5018-4e0b-995e-f93129a1ff1d None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:02.601656 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd261c0-5018-4e0b-995e-f93129a1ff1d None admin] PATH_INFO: `/v3/projects/328b693aa62e4467aa34a749006ad00f` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:02.603848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd261c0-5018-4e0b-995e-f93129a1ff1d None admin] RBAC: Authorizing `identity:get_project(project_id=328b693aa62e4467aa34a749006ad00f)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:02.606047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd261c0-5018-4e0b-995e-f93129a1ff1d None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:02.609447 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 149/289] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:02 2022] GET /identity/v3/projects/328b693aa62e4467aa34a749006ad00f => generated 302 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:02.624436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd261c0-5018-4e0b-995e-f93129a1ff1d None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:02.744588 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74be96-4186-472c-8d14-7bc797413fb6 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:02.817750 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464f10c-73fa-441c-bb29-ca2f5f9b38e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:02.818663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464f10c-73fa-441c-bb29-ca2f5f9b38e0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:02.818947 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464f10c-73fa-441c-bb29-ca2f5f9b38e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:02.819225 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464f10c-73fa-441c-bb29-ca2f5f9b38e0 None admin] PATH_INFO: `/v3/users/3cbb634531974782aee895d70907f60c` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:02.824821 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464f10c-73fa-441c-bb29-ca2f5f9b38e0 None admin] RBAC: Authorizing `identity:get_user(user_id=3cbb634531974782aee895d70907f60c)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:02.826854 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464f10c-73fa-441c-bb29-ca2f5f9b38e0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:02.828550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 141/290] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:02 2022] GET /identity/v3/users/3cbb634531974782aee895d70907f60c => generated 297 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:02.833060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd261c0-5018-4e0b-995e-f93129a1ff1d None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:02.835163 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464f10c-73fa-441c-bb29-ca2f5f9b38e0 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:02.889401 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:02.907300 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76811337-bc18-4885-85ba-d132790e6b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:02.908569 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76811337-bc18-4885-85ba-d132790e6b60 None admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:02.909054 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76811337-bc18-4885-85ba-d132790e6b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:02.909527 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76811337-bc18-4885-85ba-d132790e6b60 None admin] PATH_INFO: `/v3/projects/328b693aa62e4467aa34a749006ad00f/users/1c7eec7c2122493d9c841b67b3c9d281/roles/1a524a543caa49b286168391bb19d71e` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:02.911844 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76811337-bc18-4885-85ba-d132790e6b60 None admin] RBAC: Authorizing `identity:create_grant(project_id=328b693aa62e4467aa34a749006ad00f, user_id=1c7eec7c2122493d9c841b67b3c9d281, role_id=1a524a543caa49b286168391bb19d71e)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:02.913554 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76811337-bc18-4885-85ba-d132790e6b60 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:02.932163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 150/291] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 15:58:02 2022] PUT /identity/v3/projects/328b693aa62e4467aa34a749006ad00f/users/1c7eec7c2122493d9c841b67b3c9d281/roles/1a524a543caa49b286168391bb19d71e => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:58:02.996523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464f10c-73fa-441c-bb29-ca2f5f9b38e0 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:03.039124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d55b65-ec50-4052-b020-7a7b798969b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:03.040285 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d55b65-ec50-4052-b020-7a7b798969b8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:03.040797 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d55b65-ec50-4052-b020-7a7b798969b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:03.041270 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d55b65-ec50-4052-b020-7a7b798969b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:03.119866 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d55b65-ec50-4052-b020-7a7b798969b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:03.121698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d55b65-ec50-4052-b020-7a7b798969b8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:03.148346 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:03.231594 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 142/292] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:03.239116 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76811337-bc18-4885-85ba-d132790e6b60 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:03.296258 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:03.363517 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76811337-bc18-4885-85ba-d132790e6b60 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:03.399276 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992fa31-9a40-4126-98d7-07ed4e12977d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:03.400149 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992fa31-9a40-4126-98d7-07ed4e12977d None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:03.400434 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992fa31-9a40-4126-98d7-07ed4e12977d None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:03.400716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992fa31-9a40-4126-98d7-07ed4e12977d None admin] PATH_INFO: `/v3/projects/328b693aa62e4467aa34a749006ad00f` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:03.401946 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992fa31-9a40-4126-98d7-07ed4e12977d None admin] RBAC: Authorizing `identity:get_project(project_id=328b693aa62e4467aa34a749006ad00f)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:03.403413 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992fa31-9a40-4126-98d7-07ed4e12977d None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:03.406145 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 151/293] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:03 2022] GET /identity/v3/projects/328b693aa62e4467aa34a749006ad00f => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:03.411519 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d55b65-ec50-4052-b020-7a7b798969b8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:03.547736 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d55b65-ec50-4052-b020-7a7b798969b8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:03.572873 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:03.588617 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6a56d-607f-4692-90ec-2c446b970905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:03.589293 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6a56d-607f-4692-90ec-2c446b970905 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:03.589585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6a56d-607f-4692-90ec-2c446b970905 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:03.589863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6a56d-607f-4692-90ec-2c446b970905 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:03.590739 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6a56d-607f-4692-90ec-2c446b970905 None admin] RBAC: Adding query filter params (role.id=3ce570490865493380e6941d004136bd, scope.project.id=328b693aa62e4467aa34a749006ad00f, user.id=3cbb634531974782aee895d70907f60c) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:03.591055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6a56d-607f-4692-90ec-2c446b970905 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:03.593151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6a56d-607f-4692-90ec-2c446b970905 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:03.600141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 143/294] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:58:03 2022] GET /identity/v3/role_assignments?user.id=3cbb634531974782aee895d70907f60c&scope.project.id=328b693aa62e4467aa34a749006ad00f&role.id=3ce570490865493380e6941d004136bd => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:04.033164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b14866-170a-4fcc-bb58-f12f4110b9dd None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:04.033904 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b14866-170a-4fcc-bb58-f12f4110b9dd None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:04.034484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b14866-170a-4fcc-bb58-f12f4110b9dd None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:04.036402 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 152/295] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:04.042186 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaba0c1-4fd4-4c99-bc54-af79eeacbcbf None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:04.042779 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaba0c1-4fd4-4c99-bc54-af79eeacbcbf None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:04.043255 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaba0c1-4fd4-4c99-bc54-af79eeacbcbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:04.065701 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-acaba0c1-4fd4-4c99-bc54-af79eeacbcbf None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:04.067277 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-acaba0c1-4fd4-4c99-bc54-af79eeacbcbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:04.154504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 144/296] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:04.174570 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b14866-170a-4fcc-bb58-f12f4110b9dd None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:04.308477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b14866-170a-4fcc-bb58-f12f4110b9dd None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:04.318381 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:04.352651 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714378d5-7f25-4e48-b22f-85c0fd3efeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:04.354643 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714378d5-7f25-4e48-b22f-85c0fd3efeb8 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:04.355149 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714378d5-7f25-4e48-b22f-85c0fd3efeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:04.355632 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714378d5-7f25-4e48-b22f-85c0fd3efeb8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:04.369827 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76789e7-89d2-42b6-b4cb-d1270c829aee None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:04.370482 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76789e7-89d2-42b6-b4cb-d1270c829aee None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:04.370586 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714378d5-7f25-4e48-b22f-85c0fd3efeb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:04.371067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76789e7-89d2-42b6-b4cb-d1270c829aee None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:04.373046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 145/297] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:04.378996 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36ad7c-7e69-4cec-8659-071c043ac4cb None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:04.379234 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-714378d5-7f25-4e48-b22f-85c0fd3efeb8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:58:04.379234 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:04.379234 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:04.379234 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:04.379234 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:04.379234 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:04.379234 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:04.379234 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:04.379234 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:04.379234 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:04.379234 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:04.379234 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:04.379234 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:04.379234 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:04.379234 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:04.379234 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:04.380083 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:04.380083 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:04.380083 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:04.380083 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:04.380083 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:04.380083 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:04.380083 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:04.380083 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:04.380083 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:04.380083 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:04.380083 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:04.380083 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:04.380083 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:04.380083 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:04.380083 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:04.380083 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:04.383654 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:04.383654 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:04.383654 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:04.383654 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:04.383654 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:04.383654 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:04.383654 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:04.383654 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:04.383654 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:04.383654 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:04.383654 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:58:04.383654 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:04.383654 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 153/298] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:04.384325 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36ad7c-7e69-4cec-8659-071c043ac4cb None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:04.384325 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36ad7c-7e69-4cec-8659-071c043ac4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:04.386525 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714378d5-7f25-4e48-b22f-85c0fd3efeb8 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:04.402384 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-5f36ad7c-7e69-4cec-8659-071c043ac4cb None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:04.403644 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5f36ad7c-7e69-4cec-8659-071c043ac4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:04.489554 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 146/299] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:58:04.512097 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714378d5-7f25-4e48-b22f-85c0fd3efeb8 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:04.513898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36ad7c-7e69-4cec-8659-071c043ac4cb None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:04.553419 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf951fc1-1ad6-4ebb-a7fc-e434b04d89a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:04.554577 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf951fc1-1ad6-4ebb-a7fc-e434b04d89a3 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:04.555067 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf951fc1-1ad6-4ebb-a7fc-e434b04d89a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:04.555542 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf951fc1-1ad6-4ebb-a7fc-e434b04d89a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:04.556606 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf951fc1-1ad6-4ebb-a7fc-e434b04d89a3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:04.557198 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf951fc1-1ad6-4ebb-a7fc-e434b04d89a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:04.558900 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf951fc1-1ad6-4ebb-a7fc-e434b04d89a3 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:04.568931 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 154/300] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 15:58:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:04.574226 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:04.576748 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf951fc1-1ad6-4ebb-a7fc-e434b04d89a3 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:04.610508 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:04.638861 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36ad7c-7e69-4cec-8659-071c043ac4cb None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:04.683406 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbffb7d-92ae-4cef-b6d2-5b8ca340bba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:04.684625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbffb7d-92ae-4cef-b6d2-5b8ca340bba4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:04.685658 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbffb7d-92ae-4cef-b6d2-5b8ca340bba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:04.686212 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbffb7d-92ae-4cef-b6d2-5b8ca340bba4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:04.687600 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbffb7d-92ae-4cef-b6d2-5b8ca340bba4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:04.689810 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbffb7d-92ae-4cef-b6d2-5b8ca340bba4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:04.693973 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 147/301] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:04.698124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbffb7d-92ae-4cef-b6d2-5b8ca340bba4 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:04.702825 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf951fc1-1ad6-4ebb-a7fc-e434b04d89a3 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:04.747010 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e471be-b537-47f2-80e7-ea8f849b5ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:04.748781 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e471be-b537-47f2-80e7-ea8f849b5ada None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:04.749266 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e471be-b537-47f2-80e7-ea8f849b5ada None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:04.749736 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e471be-b537-47f2-80e7-ea8f849b5ada None admin] PATH_INFO: `/v3/users/980448110764436083689f8a16e322d1` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:04.771906 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e471be-b537-47f2-80e7-ea8f849b5ada None admin] RBAC: Authorizing `identity:get_user(user_id=980448110764436083689f8a16e322d1)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:04.773770 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e471be-b537-47f2-80e7-ea8f849b5ada None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:04.775834 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 155/302] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:04 2022] GET /identity/v3/users/980448110764436083689f8a16e322d1 => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:04.783240 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e471be-b537-47f2-80e7-ea8f849b5ada None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:04.863129 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbffb7d-92ae-4cef-b6d2-5b8ca340bba4 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:04.878930 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:04.879985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:04.903190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e471be-b537-47f2-80e7-ea8f849b5ada None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:04.909236 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b1559e-5161-4187-bbe9-30ad865748ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:04.910369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1559e-5161-4187-bbe9-30ad865748ff None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:04.910887 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1559e-5161-4187-bbe9-30ad865748ff None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:04.911429 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1559e-5161-4187-bbe9-30ad865748ff None admin] PATH_INFO: `/v3/groups` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:04.912426 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b1559e-5161-4187-bbe9-30ad865748ff None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:04.914189 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b1559e-5161-4187-bbe9-30ad865748ff None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:04.932119 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 148/303] 10.222.0.17 () {60 vars in 1134 bytes} [Thu Jun 16 15:58:04 2022] POST /identity/v3/groups => generated 229 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:58:04.950652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a382218-fd61-4e48-8599-be80c4ec2761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:04.952188 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a382218-fd61-4e48-8599-be80c4ec2761 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:04.952420 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a382218-fd61-4e48-8599-be80c4ec2761 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:04.952647 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a382218-fd61-4e48-8599-be80c4ec2761 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:04.953510 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a382218-fd61-4e48-8599-be80c4ec2761 None admin] RBAC: Adding query filter params (role.id=1a524a543caa49b286168391bb19d71e, user.id=980448110764436083689f8a16e322d1, scope.system=all) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:04.953831 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a382218-fd61-4e48-8599-be80c4ec2761 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:04.955384 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a382218-fd61-4e48-8599-be80c4ec2761 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:04.965073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 156/304] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jun 16 15:58:04 2022] GET /identity/v3/role_assignments?user.id=980448110764436083689f8a16e322d1&scope.system=all&role.id=1a524a543caa49b286168391bb19d71e => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:04.968207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732788d-66bf-4f9a-ab8c-cdfc2ce696c6 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:04.968861 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732788d-66bf-4f9a-ab8c-cdfc2ce696c6 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:04.969483 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732788d-66bf-4f9a-ab8c-cdfc2ce696c6 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:04.971404 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 149/305] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:58:04.977530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a963a52-575c-4453-9df2-f8fb6d51ae3b None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:04.977883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a963a52-575c-4453-9df2-f8fb6d51ae3b None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:04.978134 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a963a52-575c-4453-9df2-f8fb6d51ae3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:05.000414 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-1a963a52-575c-4453-9df2-f8fb6d51ae3b None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:05.001728 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-1a963a52-575c-4453-9df2-f8fb6d51ae3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:05.095145 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 157/306] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:05.118778 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732788d-66bf-4f9a-ab8c-cdfc2ce696c6 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:05.238952 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732788d-66bf-4f9a-ab8c-cdfc2ce696c6 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:05.276016 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372a139-654d-4022-b57c-a42629fd81d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:05.276828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372a139-654d-4022-b57c-a42629fd81d3 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:05.277106 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372a139-654d-4022-b57c-a42629fd81d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:05.277379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372a139-654d-4022-b57c-a42629fd81d3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:05.289578 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c372a139-654d-4022-b57c-a42629fd81d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:05.298624 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-c372a139-654d-4022-b57c-a42629fd81d3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:05.298624 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:05.298624 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:05.298624 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:05.298624 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:05.298624 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:05.298624 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:05.298624 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:05.298624 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:05.298624 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:05.298624 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:05.298624 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:05.298624 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:05.298624 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:05.298624 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:05.298624 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:05.299520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:05.299520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:05.299520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:05.299520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:05.299520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:05.299520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:05.299520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:05.299520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:05.299520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:05.299520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:05.299520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:05.299520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:05.299520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:05.299520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:05.299520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:05.299520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:05.300556 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:05.300556 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:05.300556 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:05.300556 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:05.300556 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:05.300556 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:05.300556 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:05.300556 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:05.300556 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:05.300556 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:05.300556 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:05.300556 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:05.300556 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 150/307] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:05.306236 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a963a52-575c-4453-9df2-f8fb6d51ae3b None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:05.395613 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:05.416535 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a963a52-575c-4453-9df2-f8fb6d51ae3b None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:05.455686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e142bbe-fbd8-4a6a-83bf-596c11effc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:05.456367 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e142bbe-fbd8-4a6a-83bf-596c11effc6e None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:05.456651 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e142bbe-fbd8-4a6a-83bf-596c11effc6e None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:05.456983 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e142bbe-fbd8-4a6a-83bf-596c11effc6e None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:05.457747 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e142bbe-fbd8-4a6a-83bf-596c11effc6e None admin] RBAC: Adding query filter params (name=member) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:05.458098 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e142bbe-fbd8-4a6a-83bf-596c11effc6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:05.459499 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e142bbe-fbd8-4a6a-83bf-596c11effc6e None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:05.469076 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 158/308] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 15:58:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:05.474829 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372a139-654d-4022-b57c-a42629fd81d3 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:05.514953 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:05.597640 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372a139-654d-4022-b57c-a42629fd81d3 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:05.638058 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e265e-2bf3-4efa-ade4-c9df4b21edeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:05.639366 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e265e-2bf3-4efa-ade4-c9df4b21edeb None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:05.639853 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e265e-2bf3-4efa-ade4-c9df4b21edeb None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:05.640399 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e265e-2bf3-4efa-ade4-c9df4b21edeb None admin] PATH_INFO: `/v3/users/80f3e90f5e5746f7bbf9156097c21d39` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:05.641872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e265e-2bf3-4efa-ade4-c9df4b21edeb None admin] RBAC: Authorizing `identity:get_user(user_id=80f3e90f5e5746f7bbf9156097c21d39)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:05.643570 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e265e-2bf3-4efa-ade4-c9df4b21edeb None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:05.645549 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 151/309] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:05 2022] GET /identity/v3/users/80f3e90f5e5746f7bbf9156097c21d39 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:05.650023 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf17b3-c4be-443c-82ff-0049e82a7032 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:05.650023 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf17b3-c4be-443c-82ff-0049e82a7032 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:05.650023 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf17b3-c4be-443c-82ff-0049e82a7032 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:05.650023 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 159/310] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:05.651580 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e265e-2bf3-4efa-ade4-c9df4b21edeb None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:05.658451 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f082c-9506-4220-9191-5b4701dfa0d2 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:05.658451 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f082c-9506-4220-9191-5b4701dfa0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:05.658451 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f082c-9506-4220-9191-5b4701dfa0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:05.674837 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-ab8f082c-9506-4220-9191-5b4701dfa0d2 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:05.676039 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-ab8f082c-9506-4220-9191-5b4701dfa0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:05.771214 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e265e-2bf3-4efa-ade4-c9df4b21edeb None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:05.773047 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:05.783978 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 160/311] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:58:05.808352 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f082c-9506-4220-9191-5b4701dfa0d2 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:05.823554 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7984a08-78c8-48f1-82e9-6a04e3cbd456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:05.824792 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7984a08-78c8-48f1-82e9-6a04e3cbd456 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:05.825270 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7984a08-78c8-48f1-82e9-6a04e3cbd456 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:05.827122 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7984a08-78c8-48f1-82e9-6a04e3cbd456 None admin] PATH_INFO: `/v3/system/users/80f3e90f5e5746f7bbf9156097c21d39/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:05.828925 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7984a08-78c8-48f1-82e9-6a04e3cbd456 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=80f3e90f5e5746f7bbf9156097c21d39, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:05.830825 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7984a08-78c8-48f1-82e9-6a04e3cbd456 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:05.850717 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 152/312] 10.222.0.17 () {58 vars in 1334 bytes} [Thu Jun 16 15:58:05 2022] PUT /identity/v3/system/users/80f3e90f5e5746f7bbf9156097c21d39/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:58:05.893597 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:05.927368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f082c-9506-4220-9191-5b4701dfa0d2 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:05.967193 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d82bed-7b7a-4221-b958-aa3cc652373b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:05.967940 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d82bed-7b7a-4221-b958-aa3cc652373b None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:05.968218 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d82bed-7b7a-4221-b958-aa3cc652373b None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:05.968499 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d82bed-7b7a-4221-b958-aa3cc652373b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:05.980427 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d82bed-7b7a-4221-b958-aa3cc652373b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:05.989339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-d6d82bed-7b7a-4221-b958-aa3cc652373b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:58:05.989339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:05.989339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:05.989339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:05.989339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:05.989339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:05.989339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:05.989339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:05.989339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:05.989339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:05.989339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:05.989339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:05.989339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:05.989339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:05.989339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:05.989339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:05.990318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:05.990318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:05.990318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:05.990318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:05.990318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:05.990318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:05.990318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:05.990318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:05.990318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:05.990318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:05.990318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:05.990318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:05.990318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:05.990318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:05.990318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:05.990318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:05.991118 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:05.991118 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:05.991118 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:05.991118 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:05.991118 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:05.991118 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:05.991118 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:05.991118 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:05.991118 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:05.991118 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:05.991118 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:58:05.991118 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:05.991118 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 161/313] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:58:06.000275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7984a08-78c8-48f1-82e9-6a04e3cbd456 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:06.131088 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7984a08-78c8-48f1-82e9-6a04e3cbd456 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:06.174714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458abb0b-67e5-4834-896c-66c3d3013e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:06.176381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458abb0b-67e5-4834-896c-66c3d3013e15 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:06.177139 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458abb0b-67e5-4834-896c-66c3d3013e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:06.177985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458abb0b-67e5-4834-896c-66c3d3013e15 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:06.179732 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458abb0b-67e5-4834-896c-66c3d3013e15 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:06.180691 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458abb0b-67e5-4834-896c-66c3d3013e15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:06.183834 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458abb0b-67e5-4834-896c-66c3d3013e15 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:06.198927 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 153/314] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 15:58:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:06.205706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d82bed-7b7a-4221-b958-aa3cc652373b None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:06.322058 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d82bed-7b7a-4221-b958-aa3cc652373b None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:06.353029 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:06.359006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db203421-8db0-4189-9e58-d0d7a4400dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:06.360067 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db203421-8db0-4189-9e58-d0d7a4400dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:06.360362 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db203421-8db0-4189-9e58-d0d7a4400dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:06.360644 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db203421-8db0-4189-9e58-d0d7a4400dd8 None admin] PATH_INFO: `/v3/users/1c7eec7c2122493d9c841b67b3c9d281` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:06.361908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db203421-8db0-4189-9e58-d0d7a4400dd8 None admin] RBAC: Authorizing `identity:get_user(user_id=1c7eec7c2122493d9c841b67b3c9d281)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:06.363420 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db203421-8db0-4189-9e58-d0d7a4400dd8 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:06.365467 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 162/315] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:06 2022] GET /identity/v3/users/1c7eec7c2122493d9c841b67b3c9d281 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:06.371022 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458abb0b-67e5-4834-896c-66c3d3013e15 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:06.405687 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:06.491322 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458abb0b-67e5-4834-896c-66c3d3013e15 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:06.527303 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311a812-4738-457f-8b97-16ce92591a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:06.528043 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311a812-4738-457f-8b97-16ce92591a1e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:06.528250 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311a812-4738-457f-8b97-16ce92591a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:06.528532 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311a812-4738-457f-8b97-16ce92591a1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:06.608379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8311a812-4738-457f-8b97-16ce92591a1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:06.610233 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8311a812-4738-457f-8b97-16ce92591a1e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:06.640503 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:06.722123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 154/316] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:06.727726 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db203421-8db0-4189-9e58-d0d7a4400dd8 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:06.843095 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db203421-8db0-4189-9e58-d0d7a4400dd8 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:06.879279 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a83eaa-9d1c-469e-9ee2-8cef36744c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:06.880142 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a83eaa-9d1c-469e-9ee2-8cef36744c3a None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:06.880429 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a83eaa-9d1c-469e-9ee2-8cef36744c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:06.880710 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a83eaa-9d1c-469e-9ee2-8cef36744c3a None admin] PATH_INFO: `/v3/projects/328b693aa62e4467aa34a749006ad00f` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:06.882206 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a83eaa-9d1c-469e-9ee2-8cef36744c3a None admin] RBAC: Authorizing `identity:get_project(project_id=328b693aa62e4467aa34a749006ad00f)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:06.883682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a83eaa-9d1c-469e-9ee2-8cef36744c3a None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:06.885741 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 163/317] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:06 2022] GET /identity/v3/projects/328b693aa62e4467aa34a749006ad00f => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:06.892506 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311a812-4738-457f-8b97-16ce92591a1e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:06.999587 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311a812-4738-457f-8b97-16ce92591a1e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:07.029364 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:07.038620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e70be4-a056-41d2-b0be-9307834d8f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:07.039225 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e70be4-a056-41d2-b0be-9307834d8f58 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:07.039509 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e70be4-a056-41d2-b0be-9307834d8f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:07.039784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e70be4-a056-41d2-b0be-9307834d8f58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:07.040631 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e70be4-a056-41d2-b0be-9307834d8f58 None admin] RBAC: Adding query filter params (role.id=1a524a543caa49b286168391bb19d71e, scope.project.id=328b693aa62e4467aa34a749006ad00f, user.id=1c7eec7c2122493d9c841b67b3c9d281) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:07.040945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e70be4-a056-41d2-b0be-9307834d8f58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:07.042529 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e70be4-a056-41d2-b0be-9307834d8f58 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:07.049806 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 155/318] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:58:06 2022] GET /identity/v3/role_assignments?user.id=1c7eec7c2122493d9c841b67b3c9d281&scope.project.id=328b693aa62e4467aa34a749006ad00f&role.id=1a524a543caa49b286168391bb19d71e => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:07.274275 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9769578-3b1f-4481-9d66-8dafa8d1cd4b None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:07.274817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9769578-3b1f-4481-9d66-8dafa8d1cd4b None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:07.275266 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9769578-3b1f-4481-9d66-8dafa8d1cd4b None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:07.278312 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 164/319] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:07.286012 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39555950-3ae6-4feb-9aec-e8ab755c5886 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:07.286302 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39555950-3ae6-4feb-9aec-e8ab755c5886 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:07.286584 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39555950-3ae6-4feb-9aec-e8ab755c5886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:07.312559 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-39555950-3ae6-4feb-9aec-e8ab755c5886 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:07.314161 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-39555950-3ae6-4feb-9aec-e8ab755c5886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:07.384340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc53b0e-04d0-401f-8901-8ad2c06bb9e5 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:07.384683 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc53b0e-04d0-401f-8901-8ad2c06bb9e5 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:07.385196 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc53b0e-04d0-401f-8901-8ad2c06bb9e5 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:07.386662 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 165/320] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:58:07.392501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8cb17-1c72-43e1-a9b0-008f53183862 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:07.393028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8cb17-1c72-43e1-a9b0-008f53183862 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:07.393028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8cb17-1c72-43e1-a9b0-008f53183862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:07.411008 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-c4c8cb17-1c72-43e1-a9b0-008f53183862 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:07.411192 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 156/321] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:07.412164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-c4c8cb17-1c72-43e1-a9b0-008f53183862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:07.433522 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39555950-3ae6-4feb-9aec-e8ab755c5886 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:07.480261 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:07.487893 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 166/322] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:58:07.510609 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8cb17-1c72-43e1-a9b0-008f53183862 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:07.552323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39555950-3ae6-4feb-9aec-e8ab755c5886 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:07.577759 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:07.591722 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd779ea3-2d32-47e2-9ae4-9b7d9a2c3726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:07.592533 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd779ea3-2d32-47e2-9ae4-9b7d9a2c3726 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:07.592819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd779ea3-2d32-47e2-9ae4-9b7d9a2c3726 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:07.593096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd779ea3-2d32-47e2-9ae4-9b7d9a2c3726 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:07.604458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd779ea3-2d32-47e2-9ae4-9b7d9a2c3726 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:07.612665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-bd779ea3-2d32-47e2-9ae4-9b7d9a2c3726 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:58:07.612665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:07.612665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:07.612665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:07.612665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:07.612665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:07.612665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:07.612665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:07.612665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:07.612665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:07.612665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:07.612665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:07.612665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:07.612665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:07.612665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:07.612665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:07.613744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:07.613744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:07.613744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:07.613744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:07.613744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:07.613744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:07.613744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:07.613744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:07.613744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:07.613744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:07.613744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:07.613744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:07.613744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:07.613744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:07.613744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:07.613744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:07.614619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:07.614619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:07.614619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:07.614619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:07.614619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:07.614619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:07.614619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:07.614619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:07.614619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:07.614619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:07.614619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:58:07.614619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:07.614619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 157/323] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:58:07.619986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd779ea3-2d32-47e2-9ae4-9b7d9a2c3726 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:07.627964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8cb17-1c72-43e1-a9b0-008f53183862 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:07.664892 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed439d-3848-4f2d-b029-db08b68afcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:07.665693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed439d-3848-4f2d-b029-db08b68afcce None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:07.666008 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed439d-3848-4f2d-b029-db08b68afcce None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:07.666289 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed439d-3848-4f2d-b029-db08b68afcce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:07.667469 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed439d-3848-4f2d-b029-db08b68afcce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:07.668929 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed439d-3848-4f2d-b029-db08b68afcce None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:07.671050 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 167/324] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:07.676134 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed439d-3848-4f2d-b029-db08b68afcce None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:07.736067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd779ea3-2d32-47e2-9ae4-9b7d9a2c3726 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:07.757280 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:07.772976 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382648b-4338-4c5f-ac81-b0a482e33c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:07.773662 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382648b-4338-4c5f-ac81-b0a482e33c83 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:07.773967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382648b-4338-4c5f-ac81-b0a482e33c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:07.774258 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382648b-4338-4c5f-ac81-b0a482e33c83 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:07.775017 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9382648b-4338-4c5f-ac81-b0a482e33c83 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:07.775331 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9382648b-4338-4c5f-ac81-b0a482e33c83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:07.776747 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9382648b-4338-4c5f-ac81-b0a482e33c83 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:07.785971 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 158/325] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 15:58:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:07.791408 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382648b-4338-4c5f-ac81-b0a482e33c83 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:07.811835 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed439d-3848-4f2d-b029-db08b68afcce None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:07.842396 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:07.849446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39012cb-6386-4b37-a689-77e2265cb165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:07.850067 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39012cb-6386-4b37-a689-77e2265cb165 None admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:07.850349 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39012cb-6386-4b37-a689-77e2265cb165 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:07.850622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39012cb-6386-4b37-a689-77e2265cb165 None admin] PATH_INFO: `/v3/groups` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:07.851316 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39012cb-6386-4b37-a689-77e2265cb165 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:07.852783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39012cb-6386-4b37-a689-77e2265cb165 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:07.869542 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 168/326] 10.222.0.17 () {60 vars in 1134 bytes} [Thu Jun 16 15:58:07 2022] POST /identity/v3/groups => generated 226 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:58:07.912502 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382648b-4338-4c5f-ac81-b0a482e33c83 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:07.955047 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e8cdb-e41b-42d9-81ee-eeae4588d3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:07.955933 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e8cdb-e41b-42d9-81ee-eeae4588d3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:07.956214 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e8cdb-e41b-42d9-81ee-eeae4588d3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:07.956501 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e8cdb-e41b-42d9-81ee-eeae4588d3f4 None admin] PATH_INFO: `/v3/users/980448110764436083689f8a16e322d1` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:07.957736 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e8cdb-e41b-42d9-81ee-eeae4588d3f4 None admin] RBAC: Authorizing `identity:get_user(user_id=980448110764436083689f8a16e322d1)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:07.959290 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e8cdb-e41b-42d9-81ee-eeae4588d3f4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:07.961290 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 159/327] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:07 2022] GET /identity/v3/users/980448110764436083689f8a16e322d1 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:07.967510 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39012cb-6386-4b37-a689-77e2265cb165 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:08.089302 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39012cb-6386-4b37-a689-77e2265cb165 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:08.119416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:08.130096 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ade5f4-e88b-4dbe-a204-2e0560083e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:08.130992 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ade5f4-e88b-4dbe-a204-2e0560083e0c None admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:08.131275 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ade5f4-e88b-4dbe-a204-2e0560083e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:08.131553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ade5f4-e88b-4dbe-a204-2e0560083e0c None admin] PATH_INFO: `/v3/system/users/980448110764436083689f8a16e322d1/roles/1a524a543caa49b286168391bb19d71e` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:08.134187 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ade5f4-e88b-4dbe-a204-2e0560083e0c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=980448110764436083689f8a16e322d1, role_id=1a524a543caa49b286168391bb19d71e)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:08.135742 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ade5f4-e88b-4dbe-a204-2e0560083e0c None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:08.147447 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 169/328] 10.222.0.17 () {58 vars in 1334 bytes} [Thu Jun 16 15:58:07 2022] PUT /identity/v3/system/users/980448110764436083689f8a16e322d1/roles/1a524a543caa49b286168391bb19d71e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:58:08.208984 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e201997-4d15-47a7-be3b-d42ed7ff01ae None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:08.209661 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e201997-4d15-47a7-be3b-d42ed7ff01ae None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:08.210172 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e201997-4d15-47a7-be3b-d42ed7ff01ae None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:08.212491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 160/329] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:08.218201 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ab114-f718-4cff-aaee-9f2d1c330c23 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:08.218518 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ab114-f718-4cff-aaee-9f2d1c330c23 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:08.218766 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ab114-f718-4cff-aaee-9f2d1c330c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:08.238472 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-a77ab114-f718-4cff-aaee-9f2d1c330c23 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:08.239747 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-a77ab114-f718-4cff-aaee-9f2d1c330c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:08.329355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 170/330] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:08.353267 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e201997-4d15-47a7-be3b-d42ed7ff01ae None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:08.476682 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e201997-4d15-47a7-be3b-d42ed7ff01ae None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:08.492158 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:08.517275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17719ef-7d56-4493-b985-66229dc1b1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:08.518121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17719ef-7d56-4493-b985-66229dc1b1ee None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:08.518405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17719ef-7d56-4493-b985-66229dc1b1ee None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:08.518681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17719ef-7d56-4493-b985-66229dc1b1ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:08.530779 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17719ef-7d56-4493-b985-66229dc1b1ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:08.538699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-c17719ef-7d56-4493-b985-66229dc1b1ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:08.538699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:08.538699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:08.538699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:08.538699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:08.538699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:08.538699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:08.538699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:08.538699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:08.538699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:08.538699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:08.538699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:08.538699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:08.538699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:08.538699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:08.538699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:08.539641 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:08.539641 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:08.539641 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:08.539641 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:08.539641 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:08.539641 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:08.539641 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:08.539641 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:08.539641 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:08.539641 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:08.539641 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:08.539641 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:08.539641 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:08.539641 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:08.539641 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:08.539641 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:08.540427 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:08.540427 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:08.540427 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:08.540427 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:08.540427 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:08.540427 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:08.540427 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:08.540427 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:08.540427 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:08.540427 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:08.540427 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:08.540427 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:08.540427 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 161/331] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:08.546448 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ab114-f718-4cff-aaee-9f2d1c330c23 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:08.674633 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ab114-f718-4cff-aaee-9f2d1c330c23 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:08.698632 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:08.718735 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce8054-43cf-4bcf-97c4-46bb269ef8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:08.719754 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce8054-43cf-4bcf-97c4-46bb269ef8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:08.720250 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce8054-43cf-4bcf-97c4-46bb269ef8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:08.720730 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce8054-43cf-4bcf-97c4-46bb269ef8d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:08.721686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce8054-43cf-4bcf-97c4-46bb269ef8d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:08.722222 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce8054-43cf-4bcf-97c4-46bb269ef8d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:08.723822 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce8054-43cf-4bcf-97c4-46bb269ef8d3 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:08.736717 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 171/332] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 15:58:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:08.742470 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17719ef-7d56-4493-b985-66229dc1b1ee None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:08.874367 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17719ef-7d56-4493-b985-66229dc1b1ee None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:08.920976 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676c56e4-cdd7-42d2-9c46-35e680817542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:08.921866 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c56e4-cdd7-42d2-9c46-35e680817542 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:08.922180 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c56e4-cdd7-42d2-9c46-35e680817542 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:08.922458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c56e4-cdd7-42d2-9c46-35e680817542 None admin] PATH_INFO: `/v3/users/80f3e90f5e5746f7bbf9156097c21d39` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:08.923726 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676c56e4-cdd7-42d2-9c46-35e680817542 None admin] RBAC: Authorizing `identity:get_user(user_id=80f3e90f5e5746f7bbf9156097c21d39)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:08.925184 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676c56e4-cdd7-42d2-9c46-35e680817542 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:08.926950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 162/333] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:08 2022] GET /identity/v3/users/80f3e90f5e5746f7bbf9156097c21d39 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:08.932720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce8054-43cf-4bcf-97c4-46bb269ef8d3 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:09.082018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce8054-43cf-4bcf-97c4-46bb269ef8d3 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:09.129486 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1a093-c423-4cd6-8ce2-a61449ca1a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:09.130190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1a093-c423-4cd6-8ce2-a61449ca1a05 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:09.130480 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1a093-c423-4cd6-8ce2-a61449ca1a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:09.130762 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1a093-c423-4cd6-8ce2-a61449ca1a05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:09.131616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa1a093-c423-4cd6-8ce2-a61449ca1a05 None admin] RBAC: Adding query filter params (role.id=3ce570490865493380e6941d004136bd, user.id=80f3e90f5e5746f7bbf9156097c21d39, scope.system=all) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:09.132187 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa1a093-c423-4cd6-8ce2-a61449ca1a05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:09.133676 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa1a093-c423-4cd6-8ce2-a61449ca1a05 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:09.152979 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 172/334] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jun 16 15:58:08 2022] GET /identity/v3/role_assignments?user.id=80f3e90f5e5746f7bbf9156097c21d39&scope.system=all&role.id=3ce570490865493380e6941d004136bd => generated 510 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:10.945919 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd17b7-039d-400e-9360-18b29084b971 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:10.946989 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd17b7-039d-400e-9360-18b29084b971 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:10.947487 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd17b7-039d-400e-9360-18b29084b971 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:10.949087 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 163/335] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:10.955881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027c8ed-7526-43a5-8b04-827639d2890e None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:10.956795 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027c8ed-7526-43a5-8b04-827639d2890e None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:10.957430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027c8ed-7526-43a5-8b04-827639d2890e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:10.976496 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-8027c8ed-7526-43a5-8b04-827639d2890e None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:10.978118 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-8027c8ed-7526-43a5-8b04-827639d2890e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:10.994485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a07018-49c3-487b-a610-bcf33323eebd None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:10.995485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a07018-49c3-487b-a610-bcf33323eebd None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:10.995973 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a07018-49c3-487b-a610-bcf33323eebd None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:10.999495 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 164/336] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:11.005223 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99686a7f-5aee-4902-88f8-3562e76776c8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:11.005791 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99686a7f-5aee-4902-88f8-3562e76776c8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:11.006242 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99686a7f-5aee-4902-88f8-3562e76776c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:11.042430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:11.043530 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-99686a7f-5aee-4902-88f8-3562e76776c8 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:11.045051 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-99686a7f-5aee-4902-88f8-3562e76776c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:11.104528 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 173/337] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:11.105881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfabee-10ea-43d4-a25b-9e439fedcf91 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:11.106198 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfabee-10ea-43d4-a25b-9e439fedcf91 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:11.106481 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfabee-10ea-43d4-a25b-9e439fedcf91 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:11.113546 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 174/338] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:11.114920 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4393c-fb67-4d12-a9d1-2e0fc65f9de9 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:11.115403 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4393c-fb67-4d12-a9d1-2e0fc65f9de9 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:11.115828 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4393c-fb67-4d12-a9d1-2e0fc65f9de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:11.130312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:11.156832 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-55b4393c-fb67-4d12-a9d1-2e0fc65f9de9 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:11.158133 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-55b4393c-fb67-4d12-a9d1-2e0fc65f9de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:11.166006 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 165/339] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:11.167076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99686a7f-5aee-4902-88f8-3562e76776c8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:11.254757 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 175/340] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:11.264774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4393c-fb67-4d12-a9d1-2e0fc65f9de9 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:11.334428 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99686a7f-5aee-4902-88f8-3562e76776c8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:11.375591 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9e36f-2eac-404d-8c88-67467efbfa00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:11.376416 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9e36f-2eac-404d-8c88-67467efbfa00 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:11.376696 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9e36f-2eac-404d-8c88-67467efbfa00 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:11.376967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9e36f-2eac-404d-8c88-67467efbfa00 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:11.411231 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d9e36f-2eac-404d-8c88-67467efbfa00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:11.419500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-c4d9e36f-2eac-404d-8c88-67467efbfa00 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:58:11.419500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:11.419500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:11.419500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:11.419500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:11.419500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:11.419500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:11.419500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:11.419500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:11.419500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:11.419500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:11.419500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:11.419500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:11.419500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:11.419500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:11.419500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:11.421300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:11.421300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:11.421300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:11.421300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:11.421300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:11.421300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:11.421300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:11.421300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:11.421300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:11.421300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:11.421300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:11.421300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:11.421300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:11.421300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:11.421300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:11.421300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:11.422319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:11.422319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:11.422319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:11.422319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:11.422319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:11.422319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:11.422319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:11.422319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:11.422319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:11.422319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:11.422319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:58:11.422319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:11.429582 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 166/341] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 15:58:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:11.430931 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9e36f-2eac-404d-8c88-67467efbfa00 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:11.474595 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:11.476702 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:11.479811 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4393c-fb67-4d12-a9d1-2e0fc65f9de9 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:11.581092 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda75522-c7f7-40e8-b885-731d2f73bc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:11.581903 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda75522-c7f7-40e8-b885-731d2f73bc05 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:11.582209 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda75522-c7f7-40e8-b885-731d2f73bc05 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:11.582485 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda75522-c7f7-40e8-b885-731d2f73bc05 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:11.589097 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9e36f-2eac-404d-8c88-67467efbfa00 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:11.598102 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda75522-c7f7-40e8-b885-731d2f73bc05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:11.608033 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-dda75522-c7f7-40e8-b885-731d2f73bc05 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:58:11.608033 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:11.608033 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:11.608033 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:11.608033 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:11.608033 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:11.608033 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:11.608033 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:11.608033 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:11.608033 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:11.608033 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:11.608033 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:11.608033 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:11.608033 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:11.608033 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:11.608033 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:11.609859 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:11.609859 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:11.609859 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:11.609859 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:11.609859 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:11.609859 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:11.609859 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:11.609859 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:11.609859 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:11.609859 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:11.609859 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:11.609859 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:11.609859 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:11.609859 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:11.609859 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:11.609859 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:11.610720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:11.610720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:11.610720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:11.610720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:11.610720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:11.610720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:11.610720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:11.610720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:11.610720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:11.610720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:11.610720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:58:11.610720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:11.610720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 176/342] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:11.610720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda75522-c7f7-40e8-b885-731d2f73bc05 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:11.636044 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5999ce63-77f7-4632-8569-c810dd62a69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:11.637014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999ce63-77f7-4632-8569-c810dd62a69f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:11.637300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999ce63-77f7-4632-8569-c810dd62a69f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:11.637573 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999ce63-77f7-4632-8569-c810dd62a69f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:11.651208 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5999ce63-77f7-4632-8569-c810dd62a69f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:11.662674 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-5999ce63-77f7-4632-8569-c810dd62a69f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:11.662674 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:11.662674 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:11.662674 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:11.662674 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:11.662674 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:11.662674 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:11.662674 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:11.662674 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:11.662674 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:11.662674 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:11.662674 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:11.662674 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:11.662674 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:11.662674 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:11.662674 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:11.663628 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:11.663628 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:11.663628 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:11.663628 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:11.663628 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:11.663628 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:11.663628 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:11.663628 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:11.663628 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:11.663628 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:11.663628 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:11.663628 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:11.663628 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:11.663628 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:11.663628 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:11.663628 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:11.664426 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:11.664426 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:11.664426 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:11.664426 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:11.664426 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:11.664426 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:11.664426 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:11.664426 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:11.664426 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:11.664426 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:11.664426 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:11.664426 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:11.664426 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 167/343] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:11.665643 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98374476-1c30-443d-92f5-9928d955d1a0 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:11.665643 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98374476-1c30-443d-92f5-9928d955d1a0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:11.665801 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98374476-1c30-443d-92f5-9928d955d1a0 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:11.670106 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 168/344] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:11.674646 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98374476-1c30-443d-92f5-9928d955d1a0 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:11.754521 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda75522-c7f7-40e8-b885-731d2f73bc05 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:11.801632 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c684b0-1b67-4ca2-a168-7e3b34cc1823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:11.802452 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c684b0-1b67-4ca2-a168-7e3b34cc1823 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:11.802810 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c684b0-1b67-4ca2-a168-7e3b34cc1823 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:11.803162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c684b0-1b67-4ca2-a168-7e3b34cc1823 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:11.803992 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c684b0-1b67-4ca2-a168-7e3b34cc1823 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:11.804371 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c684b0-1b67-4ca2-a168-7e3b34cc1823 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:11.806939 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c684b0-1b67-4ca2-a168-7e3b34cc1823 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:11.815475 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:11.817025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 177/345] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:11.818318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c21f71-1df9-442a-b3e5-b75cdb986e85 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:11.818700 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c21f71-1df9-442a-b3e5-b75cdb986e85 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:11.819037 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c21f71-1df9-442a-b3e5-b75cdb986e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:11.829716 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98374476-1c30-443d-92f5-9928d955d1a0 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:11.842080 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-19c21f71-1df9-442a-b3e5-b75cdb986e85 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:11.842080 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-19c21f71-1df9-442a-b3e5-b75cdb986e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:11.863698 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:11.892972 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647bccdd-dc01-44e8-ae01-01fb6ab674bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:11.893717 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647bccdd-dc01-44e8-ae01-01fb6ab674bc None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:11.894032 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647bccdd-dc01-44e8-ae01-01fb6ab674bc None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:11.894313 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647bccdd-dc01-44e8-ae01-01fb6ab674bc None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:11.895124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647bccdd-dc01-44e8-ae01-01fb6ab674bc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:11.895440 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647bccdd-dc01-44e8-ae01-01fb6ab674bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:11.896879 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647bccdd-dc01-44e8-ae01-01fb6ab674bc None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:11.907646 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 169/346] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 15:58:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:11.908606 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647bccdd-dc01-44e8-ae01-01fb6ab674bc None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:11.937737 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 178/347] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:11.939227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c21f71-1df9-442a-b3e5-b75cdb986e85 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:12.033644 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647bccdd-dc01-44e8-ae01-01fb6ab674bc None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:12.074603 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c21f71-1df9-442a-b3e5-b75cdb986e85 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:12.081164 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099ac5b-8e13-4a31-bbdd-006767d2ed14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:12.081871 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099ac5b-8e13-4a31-bbdd-006767d2ed14 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:12.082275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099ac5b-8e13-4a31-bbdd-006767d2ed14 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:12.082559 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099ac5b-8e13-4a31-bbdd-006767d2ed14 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:12.083332 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0099ac5b-8e13-4a31-bbdd-006767d2ed14 None admin] RBAC: Adding query filter params (name=member) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:12.083642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0099ac5b-8e13-4a31-bbdd-006767d2ed14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:12.085049 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0099ac5b-8e13-4a31-bbdd-006767d2ed14 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:12.095131 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 170/348] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 15:58:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:12.096001 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099ac5b-8e13-4a31-bbdd-006767d2ed14 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:12.120097 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:12.135268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900294da-c20b-42f4-ab8a-487cc573986b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:12.136304 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900294da-c20b-42f4-ab8a-487cc573986b None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:12.136790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900294da-c20b-42f4-ab8a-487cc573986b None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:12.137269 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900294da-c20b-42f4-ab8a-487cc573986b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:12.169797 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:12.230017 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900294da-c20b-42f4-ab8a-487cc573986b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:12.231544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900294da-c20b-42f4-ab8a-487cc573986b None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:12.233542 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099ac5b-8e13-4a31-bbdd-006767d2ed14 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:12.272813 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024c7ba-d893-40c8-aaf8-68028ad21a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:12.274153 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024c7ba-d893-40c8-aaf8-68028ad21a32 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:12.274668 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024c7ba-d893-40c8-aaf8-68028ad21a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:12.275151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024c7ba-d893-40c8-aaf8-68028ad21a32 None admin] PATH_INFO: `/v3/users/980448110764436083689f8a16e322d1` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:12.276580 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3024c7ba-d893-40c8-aaf8-68028ad21a32 None admin] RBAC: Authorizing `identity:get_user(user_id=980448110764436083689f8a16e322d1)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:12.278366 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3024c7ba-d893-40c8-aaf8-68028ad21a32 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:12.281255 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 171/349] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:12 2022] GET /identity/v3/users/980448110764436083689f8a16e322d1 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:12.282422 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024c7ba-d893-40c8-aaf8-68028ad21a32 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:12.342912 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 179/350] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:12.343383 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900294da-c20b-42f4-ab8a-487cc573986b None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:12.384486 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:12.401722 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:12.413063 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024c7ba-d893-40c8-aaf8-68028ad21a32 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:12.456441 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe9cf62-a88e-4a09-90bd-5661f53c2020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:12.457705 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe9cf62-a88e-4a09-90bd-5661f53c2020 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:12.458310 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe9cf62-a88e-4a09-90bd-5661f53c2020 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:12.458789 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe9cf62-a88e-4a09-90bd-5661f53c2020 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:12.459902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900294da-c20b-42f4-ab8a-487cc573986b None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:12.472146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe9cf62-a88e-4a09-90bd-5661f53c2020 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:12.480888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-6fe9cf62-a88e-4a09-90bd-5661f53c2020 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:12.480888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:12.480888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:12.480888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:12.480888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:12.480888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:12.480888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:12.480888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:12.480888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:12.480888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:12.480888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:12.480888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:12.480888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:12.480888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:12.480888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:12.480888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:12.481821 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:12.481821 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:12.481821 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:12.481821 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:12.481821 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:12.481821 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:12.481821 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:12.481821 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:12.481821 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:12.481821 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:12.481821 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:12.481821 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:12.481821 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:12.481821 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:12.481821 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:12.481821 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:12.483657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:12.483657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:12.483657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:12.483657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:12.483657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:12.483657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:12.483657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:12.483657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:12.483657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:12.483657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:12.483657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:12.483657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:12.483657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 172/351] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:12.483657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe9cf62-a88e-4a09-90bd-5661f53c2020 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:12.502870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a772e-730b-4c80-972e-f88d7d227033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:12.503094 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a772e-730b-4c80-972e-f88d7d227033 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:12.503523 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a772e-730b-4c80-972e-f88d7d227033 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:12.504080 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a772e-730b-4c80-972e-f88d7d227033 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:12.615841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a772e-730b-4c80-972e-f88d7d227033 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:12.617732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a772e-730b-4c80-972e-f88d7d227033 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:12.641795 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe9cf62-a88e-4a09-90bd-5661f53c2020 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:12.694382 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a1f08-6ef8-4a1d-abc7-916a7c42eab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:12.695065 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a1f08-6ef8-4a1d-abc7-916a7c42eab4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:12.695415 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a1f08-6ef8-4a1d-abc7-916a7c42eab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:12.696481 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:12.697179 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a1f08-6ef8-4a1d-abc7-916a7c42eab4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:12.698034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a1f08-6ef8-4a1d-abc7-916a7c42eab4 None admin] RBAC: Adding query filter params (role.id=1a524a543caa49b286168391bb19d71e, user.id=980448110764436083689f8a16e322d1, scope.system=all) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:12.698376 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a1f08-6ef8-4a1d-abc7-916a7c42eab4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:12.701614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a1f08-6ef8-4a1d-abc7-916a7c42eab4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:12.715003 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:12.727151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 173/352] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jun 16 15:58:12 2022] GET /identity/v3/role_assignments?user.id=980448110764436083689f8a16e322d1&scope.system=all&role.id=1a524a543caa49b286168391bb19d71e => generated 510 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:12.728122 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a1f08-6ef8-4a1d-abc7-916a7c42eab4 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:12.776909 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 180/353] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:12.781666 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a23c8-da57-48a6-aec3-a267e7e22a0e None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:12.781987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a23c8-da57-48a6-aec3-a267e7e22a0e None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:12.782344 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a23c8-da57-48a6-aec3-a267e7e22a0e None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:12.793952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 181/354] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:12.793952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a23c8-da57-48a6-aec3-a267e7e22a0e None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:12.909587 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a1f08-6ef8-4a1d-abc7-916a7c42eab4 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:12.946982 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a23c8-da57-48a6-aec3-a267e7e22a0e None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:12.966475 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f6a64-4583-4631-a639-d055ae1a6f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:12.968588 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f6a64-4583-4631-a639-d055ae1a6f16 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:12.969260 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f6a64-4583-4631-a639-d055ae1a6f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:12.974388 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f6a64-4583-4631-a639-d055ae1a6f16 None admin] PATH_INFO: `/v3/projects/a83dc386fda8411eb2d4aef8e43b7065` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:12.977585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f6a64-4583-4631-a639-d055ae1a6f16 None admin] RBAC: Authorizing `identity:get_project(project_id=a83dc386fda8411eb2d4aef8e43b7065)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:12.977585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f6a64-4583-4631-a639-d055ae1a6f16 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:12.979432 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 174/355] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:12 2022] GET /identity/v3/projects/a83dc386fda8411eb2d4aef8e43b7065 => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:12.980744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541f9e4-f082-464d-94a7-aa8f2d0f72e5 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:12.981033 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541f9e4-f082-464d-94a7-aa8f2d0f72e5 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:12.981308 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541f9e4-f082-464d-94a7-aa8f2d0f72e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:13.008288 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f8e28-f8aa-4be1-8c99-f7dea0633519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:13.009299 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f8e28-f8aa-4be1-8c99-f7dea0633519 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:13.009588 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f8e28-f8aa-4be1-8c99-f7dea0633519 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:13.009860 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f8e28-f8aa-4be1-8c99-f7dea0633519 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:13.010699 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f8e28-f8aa-4be1-8c99-f7dea0633519 None admin] RBAC: Adding query filter params (name=member) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:13.011009 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f8e28-f8aa-4be1-8c99-f7dea0633519 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:13.012681 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f8e28-f8aa-4be1-8c99-f7dea0633519 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:13.013266 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b541f9e4-f082-464d-94a7-aa8f2d0f72e5 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:13.015270 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b541f9e4-f082-464d-94a7-aa8f2d0f72e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:13.026706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 182/356] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 15:58:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:13.028553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f8e28-f8aa-4be1-8c99-f7dea0633519 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:13.053884 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:13.121595 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:13.129333 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 175/357] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:13.133965 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541f9e4-f082-464d-94a7-aa8f2d0f72e5 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:13.198925 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f8e28-f8aa-4be1-8c99-f7dea0633519 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:13.246837 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ad6d1-f630-4fa5-89ba-1f2cce2beaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:13.247761 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ad6d1-f630-4fa5-89ba-1f2cce2beaa8 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:13.248045 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ad6d1-f630-4fa5-89ba-1f2cce2beaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:13.248324 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ad6d1-f630-4fa5-89ba-1f2cce2beaa8 None admin] PATH_INFO: `/v3/projects/a83dc386fda8411eb2d4aef8e43b7065` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:13.250379 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ad6d1-f630-4fa5-89ba-1f2cce2beaa8 None admin] RBAC: Authorizing `identity:get_project(project_id=a83dc386fda8411eb2d4aef8e43b7065)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:13.251871 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ad6d1-f630-4fa5-89ba-1f2cce2beaa8 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:13.253648 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 183/358] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:13 2022] GET /identity/v3/projects/a83dc386fda8411eb2d4aef8e43b7065 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:13.257962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ad6d1-f630-4fa5-89ba-1f2cce2beaa8 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:13.292386 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541f9e4-f082-464d-94a7-aa8f2d0f72e5 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:13.360783 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9444313-e6cd-4fdd-bb10-1f4c7f0116e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:13.360783 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9444313-e6cd-4fdd-bb10-1f4c7f0116e6 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:13.360783 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9444313-e6cd-4fdd-bb10-1f4c7f0116e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:13.360783 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9444313-e6cd-4fdd-bb10-1f4c7f0116e6 None admin] PATH_INFO: `/v3/groups/ee669194886e4a1c9d35bf7d4fb62d91` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:13.367877 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9444313-e6cd-4fdd-bb10-1f4c7f0116e6 None admin] RBAC: Authorizing `identity:get_group(group_id=ee669194886e4a1c9d35bf7d4fb62d91)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:13.369370 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9444313-e6cd-4fdd-bb10-1f4c7f0116e6 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:13.371243 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 176/359] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 15:58:13 2022] GET /identity/v3/groups/ee669194886e4a1c9d35bf7d4fb62d91 => generated 226 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:13.372670 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a70e8-713e-4035-b1f1-465c89620ae4 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:13.374243 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a70e8-713e-4035-b1f1-465c89620ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:13.374621 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a70e8-713e-4035-b1f1-465c89620ae4 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:13.375324 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:13.376585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 177/360] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:13.378326 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a70e8-713e-4035-b1f1-465c89620ae4 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:13.415979 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ad6d1-f630-4fa5-89ba-1f2cce2beaa8 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:13.473653 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e29080-6000-473e-b58d-c2c741538ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:13.474313 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e29080-6000-473e-b58d-c2c741538ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:13.474596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e29080-6000-473e-b58d-c2c741538ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:13.474979 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e29080-6000-473e-b58d-c2c741538ec4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:13.480849 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:13.542610 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a70e8-713e-4035-b1f1-465c89620ae4 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:13.563917 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e29080-6000-473e-b58d-c2c741538ec4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:13.565416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e29080-6000-473e-b58d-c2c741538ec4 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:13.586156 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55374c5a-6590-4573-84e3-d14d0007b5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:13.586975 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55374c5a-6590-4573-84e3-d14d0007b5df None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:13.587266 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55374c5a-6590-4573-84e3-d14d0007b5df None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:13.587537 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55374c5a-6590-4573-84e3-d14d0007b5df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:13.601782 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55374c5a-6590-4573-84e3-d14d0007b5df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:13.611096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-55374c5a-6590-4573-84e3-d14d0007b5df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:58:13.611096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:13.611096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:13.611096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:13.611096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:13.611096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:13.611096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:13.611096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:13.611096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:13.611096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:13.611096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:13.611096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:13.611096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:13.611096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:13.611096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:13.611096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:13.612124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:13.612124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:13.612124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:13.612124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:13.612124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:13.612124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:13.612124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:13.612124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:13.612124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:13.612124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:13.612124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:13.612124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:13.612124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:13.612124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:13.612124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:13.612124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:13.612985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:13.612985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:13.612985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:13.612985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:13.612985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:13.612985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:13.612985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:13.612985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:13.612985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:13.612985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:13.612985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:58:13.612985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:13.612985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 178/361] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 15:58:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:13.613732 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55374c5a-6590-4573-84e3-d14d0007b5df None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:13.682267 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:13.708615 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 184/362] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:13.709572 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e29080-6000-473e-b58d-c2c741538ec4 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:13.739896 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55374c5a-6590-4573-84e3-d14d0007b5df None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:13.768494 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:13.785887 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd2493-42a4-4f2c-aab1-f3aac178ea77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:13.785887 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd2493-42a4-4f2c-aab1-f3aac178ea77 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:13.785887 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd2493-42a4-4f2c-aab1-f3aac178ea77 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:13.785887 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd2493-42a4-4f2c-aab1-f3aac178ea77 None admin] PATH_INFO: `/v3/groups/ee669194886e4a1c9d35bf7d4fb62d91` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:13.786323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bd2493-42a4-4f2c-aab1-f3aac178ea77 None admin] RBAC: Authorizing `identity:get_group(group_id=ee669194886e4a1c9d35bf7d4fb62d91)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:13.786755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bd2493-42a4-4f2c-aab1-f3aac178ea77 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:13.788725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 179/363] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 15:58:13 2022] GET /identity/v3/groups/ee669194886e4a1c9d35bf7d4fb62d91 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:13.791015 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1691d94-3a93-4297-9ba3-12bcb847d653 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:13.791627 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1691d94-3a93-4297-9ba3-12bcb847d653 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:13.791627 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1691d94-3a93-4297-9ba3-12bcb847d653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:13.811649 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-a1691d94-3a93-4297-9ba3-12bcb847d653 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:13.812877 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a1691d94-3a93-4297-9ba3-12bcb847d653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:13.846275 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e29080-6000-473e-b58d-c2c741538ec4 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:13.887212 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb181f15-39b0-4c25-8ef9-a312c89e1745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:13.887876 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb181f15-39b0-4c25-8ef9-a312c89e1745 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:13.888160 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb181f15-39b0-4c25-8ef9-a312c89e1745 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:13.888569 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb181f15-39b0-4c25-8ef9-a312c89e1745 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:13.889400 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb181f15-39b0-4c25-8ef9-a312c89e1745 None admin] RBAC: Adding query filter params (group.id=ee669194886e4a1c9d35bf7d4fb62d91, role.id=e0b0c216ef51441ab34b6e036da09daa, scope.project.id=a83dc386fda8411eb2d4aef8e43b7065) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:13.889714 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb181f15-39b0-4c25-8ef9-a312c89e1745 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:13.892493 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb181f15-39b0-4c25-8ef9-a312c89e1745 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:13.898793 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 185/364] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jun 16 15:58:13 2022] GET /identity/v3/role_assignments?group.id=ee669194886e4a1c9d35bf7d4fb62d91&scope.project.id=a83dc386fda8411eb2d4aef8e43b7065&role.id=e0b0c216ef51441ab34b6e036da09daa => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:13.899569 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb181f15-39b0-4c25-8ef9-a312c89e1745 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:13.908252 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 180/365] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:13.909443 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1691d94-3a93-4297-9ba3-12bcb847d653 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:13.975892 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:14.042057 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1691d94-3a93-4297-9ba3-12bcb847d653 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:14.053193 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb181f15-39b0-4c25-8ef9-a312c89e1745 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:14.071096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:14.096303 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7cef43-1e72-4288-bfd6-238aa4e59a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:14.097179 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7cef43-1e72-4288-bfd6-238aa4e59a19 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:14.097465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7cef43-1e72-4288-bfd6-238aa4e59a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:14.097748 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7cef43-1e72-4288-bfd6-238aa4e59a19 None admin] PATH_INFO: `/v3/projects/328b693aa62e4467aa34a749006ad00f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:14.099162 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7cef43-1e72-4288-bfd6-238aa4e59a19 None admin] RBAC: Authorizing `identity:get_project(project_id=328b693aa62e4467aa34a749006ad00f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:14.100633 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7cef43-1e72-4288-bfd6-238aa4e59a19 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:14.102480 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 181/366] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:13 2022] GET /identity/v3/projects/328b693aa62e4467aa34a749006ad00f => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:14.103336 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7cef43-1e72-4288-bfd6-238aa4e59a19 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:14.103450 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14bcb0b-d659-4046-b11d-b987882934a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:14.104179 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14bcb0b-d659-4046-b11d-b987882934a5 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:14.104467 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14bcb0b-d659-4046-b11d-b987882934a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:14.104800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14bcb0b-d659-4046-b11d-b987882934a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:14.105556 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14bcb0b-d659-4046-b11d-b987882934a5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:14.105820 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14bcb0b-d659-4046-b11d-b987882934a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:14.109151 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14bcb0b-d659-4046-b11d-b987882934a5 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:14.119159 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 186/367] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:14.119813 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14bcb0b-d659-4046-b11d-b987882934a5 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:14.250036 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7cef43-1e72-4288-bfd6-238aa4e59a19 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:14.262942 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14bcb0b-d659-4046-b11d-b987882934a5 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:14.276981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:14.300696 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816e30e-65ae-4272-9520-dcde61e92b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:14.301378 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816e30e-65ae-4272-9520-dcde61e92b83 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:14.301662 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816e30e-65ae-4272-9520-dcde61e92b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:14.301955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816e30e-65ae-4272-9520-dcde61e92b83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:14.304356 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d816e30e-65ae-4272-9520-dcde61e92b83 None admin] RBAC: Adding query filter params (group.id=ee669194886e4a1c9d35bf7d4fb62d91, role.id=3ce570490865493380e6941d004136bd, scope.project.id=a83dc386fda8411eb2d4aef8e43b7065) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:14.304800 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d816e30e-65ae-4272-9520-dcde61e92b83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:14.306602 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d816e30e-65ae-4272-9520-dcde61e92b83 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:14.313828 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4bfc3-5e89-4e52-aeef-a6cde28bb647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:14.314433 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 182/368] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jun 16 15:58:14 2022] GET /identity/v3/role_assignments?group.id=ee669194886e4a1c9d35bf7d4fb62d91&scope.project.id=a83dc386fda8411eb2d4aef8e43b7065&role.id=3ce570490865493380e6941d004136bd => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:14.315358 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4bfc3-5e89-4e52-aeef-a6cde28bb647 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:14.315358 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4bfc3-5e89-4e52-aeef-a6cde28bb647 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:14.315550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816e30e-65ae-4272-9520-dcde61e92b83 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:14.315625 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4bfc3-5e89-4e52-aeef-a6cde28bb647 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:14.331114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a4bfc3-5e89-4e52-aeef-a6cde28bb647 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:14.339564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-f6a4bfc3-5e89-4e52-aeef-a6cde28bb647 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:14.339564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:14.339564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:14.339564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:14.339564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:14.339564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:14.339564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:14.339564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:14.339564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:14.339564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:14.339564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:14.339564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:14.339564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:14.339564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:14.339564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:14.339564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:14.340511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:14.340511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:14.340511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:14.340511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:14.340511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:14.340511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:14.340511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:14.340511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:14.340511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:14.340511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:14.340511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:14.340511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:14.340511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:14.340511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:14.340511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:14.340511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:14.341301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:14.341301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:14.341301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:14.341301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:14.341301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:14.341301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:14.341301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:14.341301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:14.341301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:14.341301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:14.341301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:14.341301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:14.341301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 187/369] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 15:58:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:14.343015 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4bfc3-5e89-4e52-aeef-a6cde28bb647 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:14.395691 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:14.453544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816e30e-65ae-4272-9520-dcde61e92b83 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:14.477780 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4bfc3-5e89-4e52-aeef-a6cde28bb647 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:14.532206 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0853df1-fda3-43cd-bfa7-894a64d56e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:14.533105 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0853df1-fda3-43cd-bfa7-894a64d56e41 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:14.533392 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0853df1-fda3-43cd-bfa7-894a64d56e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:14.533669 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0853df1-fda3-43cd-bfa7-894a64d56e41 None admin] PATH_INFO: `/v3/groups/ee669194886e4a1c9d35bf7d4fb62d91` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:14.534993 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0853df1-fda3-43cd-bfa7-894a64d56e41 None admin] RBAC: Authorizing `identity:get_group(group_id=ee669194886e4a1c9d35bf7d4fb62d91)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:14.536743 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0853df1-fda3-43cd-bfa7-894a64d56e41 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:14.538843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e32d1-266b-492b-8438-d32930552081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:14.539037 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 183/370] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 15:58:14 2022] GET /identity/v3/groups/ee669194886e4a1c9d35bf7d4fb62d91 => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:14.539716 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0853df1-fda3-43cd-bfa7-894a64d56e41 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:14.540810 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e32d1-266b-492b-8438-d32930552081 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:14.541593 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e32d1-266b-492b-8438-d32930552081 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:14.546617 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e32d1-266b-492b-8438-d32930552081 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:14.617687 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:14.678212 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e32d1-266b-492b-8438-d32930552081 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:14.678431 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0853df1-fda3-43cd-bfa7-894a64d56e41 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:14.680923 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e32d1-266b-492b-8438-d32930552081 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:14.704529 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:14.727141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6528720-423f-40f5-984a-bb32d1a34424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:14.727828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6528720-423f-40f5-984a-bb32d1a34424 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:14.728114 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6528720-423f-40f5-984a-bb32d1a34424 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:14.728393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6528720-423f-40f5-984a-bb32d1a34424 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:14.729262 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6528720-423f-40f5-984a-bb32d1a34424 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:14.729575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6528720-423f-40f5-984a-bb32d1a34424 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:14.731398 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6528720-423f-40f5-984a-bb32d1a34424 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:14.742625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 184/371] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:14.747537 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6528720-423f-40f5-984a-bb32d1a34424 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:14.869634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 188/372] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:14.871396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e32d1-266b-492b-8438-d32930552081 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:14.925061 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6528720-423f-40f5-984a-bb32d1a34424 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:14.968646 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05cab0c-d021-4bbf-8d34-e0984e5e0484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:14.969307 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05cab0c-d021-4bbf-8d34-e0984e5e0484 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:14.969758 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05cab0c-d021-4bbf-8d34-e0984e5e0484 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:14.970071 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05cab0c-d021-4bbf-8d34-e0984e5e0484 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:14.971244 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05cab0c-d021-4bbf-8d34-e0984e5e0484 None admin] RBAC: Adding query filter params (group.id=ee669194886e4a1c9d35bf7d4fb62d91, role.id=3ce570490865493380e6941d004136bd, scope.project.id=328b693aa62e4467aa34a749006ad00f) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:14.971557 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05cab0c-d021-4bbf-8d34-e0984e5e0484 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:14.973276 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05cab0c-d021-4bbf-8d34-e0984e5e0484 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:14.979746 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 185/373] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jun 16 15:58:14 2022] GET /identity/v3/role_assignments?group.id=ee669194886e4a1c9d35bf7d4fb62d91&scope.project.id=328b693aa62e4467aa34a749006ad00f&role.id=3ce570490865493380e6941d004136bd => generated 260 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:14.980541 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05cab0c-d021-4bbf-8d34-e0984e5e0484 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:15.002339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:15.039606 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:15.113888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05cab0c-d021-4bbf-8d34-e0984e5e0484 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:15.119267 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e32d1-266b-492b-8438-d32930552081 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:15.164998 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1117705-72c5-4625-8f4d-9cb28ade3a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:15.165822 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1117705-72c5-4625-8f4d-9cb28ade3a7f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:15.166121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1117705-72c5-4625-8f4d-9cb28ade3a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:15.166389 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1117705-72c5-4625-8f4d-9cb28ade3a7f None admin] PATH_INFO: `/v3/projects/328b693aa62e4467aa34a749006ad00f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:15.169329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1117705-72c5-4625-8f4d-9cb28ade3a7f None admin] RBAC: Authorizing `identity:get_project(project_id=328b693aa62e4467aa34a749006ad00f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:15.172173 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1117705-72c5-4625-8f4d-9cb28ade3a7f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:15.180953 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 186/374] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:14 2022] GET /identity/v3/projects/328b693aa62e4467aa34a749006ad00f => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:15.181916 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1117705-72c5-4625-8f4d-9cb28ade3a7f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:15.219198 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada95f6-f6ac-42f9-a20c-c26604548456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:15.219882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada95f6-f6ac-42f9-a20c-c26604548456 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:15.220161 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada95f6-f6ac-42f9-a20c-c26604548456 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:15.220596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada95f6-f6ac-42f9-a20c-c26604548456 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:15.369949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1117705-72c5-4625-8f4d-9cb28ade3a7f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:15.380128 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada95f6-f6ac-42f9-a20c-c26604548456 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:15.381709 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada95f6-f6ac-42f9-a20c-c26604548456 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:15.390354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:15.411538 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e345db-e81f-412f-a86c-df1970001f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:15.412656 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e345db-e81f-412f-a86c-df1970001f8a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:15.412934 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e345db-e81f-412f-a86c-df1970001f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:15.413207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e345db-e81f-412f-a86c-df1970001f8a None admin] PATH_INFO: `/v3/groups/ee669194886e4a1c9d35bf7d4fb62d91` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:15.414410 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e345db-e81f-412f-a86c-df1970001f8a None admin] RBAC: Authorizing `identity:get_group(group_id=ee669194886e4a1c9d35bf7d4fb62d91)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:15.416422 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e345db-e81f-412f-a86c-df1970001f8a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:15.418184 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 187/375] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 15:58:15 2022] GET /identity/v3/groups/ee669194886e4a1c9d35bf7d4fb62d91 => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:15.425391 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e345db-e81f-412f-a86c-df1970001f8a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:15.511294 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:15.556452 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e345db-e81f-412f-a86c-df1970001f8a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:15.573946 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 189/376] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 704 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:15.580022 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada95f6-f6ac-42f9-a20c-c26604548456 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:15.600470 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b194e47-ca6b-47db-b4b3-3f0dfba677d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:15.601150 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b194e47-ca6b-47db-b4b3-3f0dfba677d5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:15.601441 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b194e47-ca6b-47db-b4b3-3f0dfba677d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:15.601725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b194e47-ca6b-47db-b4b3-3f0dfba677d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:15.602601 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b194e47-ca6b-47db-b4b3-3f0dfba677d5 None admin] RBAC: Adding query filter params (group.id=ee669194886e4a1c9d35bf7d4fb62d91, role.id=e0b0c216ef51441ab34b6e036da09daa, scope.project.id=328b693aa62e4467aa34a749006ad00f) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:15.602916 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b194e47-ca6b-47db-b4b3-3f0dfba677d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:15.604389 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b194e47-ca6b-47db-b4b3-3f0dfba677d5 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:15.617150 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 188/377] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jun 16 15:58:15 2022] GET /identity/v3/role_assignments?group.id=ee669194886e4a1c9d35bf7d4fb62d91&scope.project.id=328b693aa62e4467aa34a749006ad00f&role.id=e0b0c216ef51441ab34b6e036da09daa => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:15.751763 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada95f6-f6ac-42f9-a20c-c26604548456 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:15.806800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615f7da-0b40-4cea-81a9-3ea14e10a9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:15.807699 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615f7da-0b40-4cea-81a9-3ea14e10a9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:15.807983 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615f7da-0b40-4cea-81a9-3ea14e10a9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:15.808263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615f7da-0b40-4cea-81a9-3ea14e10a9a1 None admin] PATH_INFO: `/v3/projects/47ead53fd5cc424c8de9383e832d40a2` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:15.809638 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3615f7da-0b40-4cea-81a9-3ea14e10a9a1 None admin] RBAC: Authorizing `identity:get_project(project_id=47ead53fd5cc424c8de9383e832d40a2)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:15.811746 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3615f7da-0b40-4cea-81a9-3ea14e10a9a1 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:15.814087 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 190/378] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:15 2022] GET /identity/v3/projects/47ead53fd5cc424c8de9383e832d40a2 => generated 344 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:15.823115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b194e47-ca6b-47db-b4b3-3f0dfba677d5 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:15.894032 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:15.971109 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b194e47-ca6b-47db-b4b3-3f0dfba677d5 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:16.025071 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5a1f3-d806-4e1d-9c27-9a0898df8803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:16.025858 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5a1f3-d806-4e1d-9c27-9a0898df8803 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:16.026175 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5a1f3-d806-4e1d-9c27-9a0898df8803 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:16.026448 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5a1f3-d806-4e1d-9c27-9a0898df8803 None admin] PATH_INFO: `/v3/groups/15b9a6ed0fdb440e93c60902d1603904` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:16.038635 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5a1f3-d806-4e1d-9c27-9a0898df8803 None admin] RBAC: Authorizing `identity:get_group(group_id=15b9a6ed0fdb440e93c60902d1603904)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:16.040113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5a1f3-d806-4e1d-9c27-9a0898df8803 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:16.041840 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 189/379] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 15:58:15 2022] GET /identity/v3/groups/15b9a6ed0fdb440e93c60902d1603904 => generated 229 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:16.048655 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615f7da-0b40-4cea-81a9-3ea14e10a9a1 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:16.105331 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:16.196373 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615f7da-0b40-4cea-81a9-3ea14e10a9a1 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:16.249846 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e0f89-0d4a-4ac6-8830-e0229edd8820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:16.250533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e0f89-0d4a-4ac6-8830-e0229edd8820 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:16.250815 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e0f89-0d4a-4ac6-8830-e0229edd8820 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:16.251090 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e0f89-0d4a-4ac6-8830-e0229edd8820 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:16.251983 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e0f89-0d4a-4ac6-8830-e0229edd8820 None admin] RBAC: Adding query filter params (group.id=15b9a6ed0fdb440e93c60902d1603904, role.id=bbbc77ec645d4ff5a8bf29ce0377f578, scope.project.id=47ead53fd5cc424c8de9383e832d40a2) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:16.252294 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e0f89-0d4a-4ac6-8830-e0229edd8820 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:16.253789 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e0f89-0d4a-4ac6-8830-e0229edd8820 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:16.262125 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 191/380] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jun 16 15:58:16 2022] GET /identity/v3/role_assignments?group.id=15b9a6ed0fdb440e93c60902d1603904&scope.project.id=47ead53fd5cc424c8de9383e832d40a2&role.id=bbbc77ec645d4ff5a8bf29ce0377f578 => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:16.822129 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48dc28-a3a2-405a-890f-32ddaee2534f None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:16.822757 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48dc28-a3a2-405a-890f-32ddaee2534f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:16.822757 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48dc28-a3a2-405a-890f-32ddaee2534f None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:16.824410 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 190/381] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:16.825591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f1aa0-1ce7-4897-9bfb-910fd1a9fea7 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:16.826093 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f1aa0-1ce7-4897-9bfb-910fd1a9fea7 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:16.826245 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f1aa0-1ce7-4897-9bfb-910fd1a9fea7 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:16.827697 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 192/382] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:16.832904 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af375173-3a54-4b4c-ad24-54cb6d57d94f None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:16.833193 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af375173-3a54-4b4c-ad24-54cb6d57d94f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:16.833475 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af375173-3a54-4b4c-ad24-54cb6d57d94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:16.836683 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b096c95-cc6d-410f-984d-064a24999746 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:16.837258 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b096c95-cc6d-410f-984d-064a24999746 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:16.837705 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b096c95-cc6d-410f-984d-064a24999746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:16.859652 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-af375173-3a54-4b4c-ad24-54cb6d57d94f None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:16.861116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-af375173-3a54-4b4c-ad24-54cb6d57d94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:16.862943 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-8b096c95-cc6d-410f-984d-064a24999746 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:16.864146 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-8b096c95-cc6d-410f-984d-064a24999746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:16.972601 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 191/383] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:16.975169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 193/384] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:58:17.017784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af375173-3a54-4b4c-ad24-54cb6d57d94f None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:17.019299 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b096c95-cc6d-410f-984d-064a24999746 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:17.081096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:17.095745 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:17.152058 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af375173-3a54-4b4c-ad24-54cb6d57d94f None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:17.191956 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e2b39-ff49-4b8e-854a-d1e35c2c1824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:17.192774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e2b39-ff49-4b8e-854a-d1e35c2c1824 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:17.193059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e2b39-ff49-4b8e-854a-d1e35c2c1824 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:17.193331 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e2b39-ff49-4b8e-854a-d1e35c2c1824 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:17.209713 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611e2b39-ff49-4b8e-854a-d1e35c2c1824 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:17.218917 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-611e2b39-ff49-4b8e-854a-d1e35c2c1824 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:17.218917 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:17.218917 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:17.218917 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:17.218917 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:17.218917 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:17.218917 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:17.218917 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:17.218917 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:17.218917 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:17.218917 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:17.218917 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:17.218917 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:17.218917 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:17.218917 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:17.218917 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:17.219886 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:17.219886 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:17.219886 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:17.219886 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:17.219886 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:17.219886 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:17.219886 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:17.219886 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:17.219886 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:17.219886 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:17.219886 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:17.219886 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:17.219886 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:17.219886 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:17.219886 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:17.219886 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:17.220675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:17.220675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:17.220675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:17.220675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:17.220675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:17.220675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:17.220675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:17.220675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:17.220675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:17.220675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:17.220675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:17.220675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:17.228343 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 192/385] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:17.230709 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e2b39-ff49-4b8e-854a-d1e35c2c1824 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:17.294047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b096c95-cc6d-410f-984d-064a24999746 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:17.354313 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad2689-7950-431c-88bb-213d03027f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:17.355437 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:17.356684 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad2689-7950-431c-88bb-213d03027f5c None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:17.357855 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad2689-7950-431c-88bb-213d03027f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:17.358512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad2689-7950-431c-88bb-213d03027f5c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:17.360232 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e2b39-ff49-4b8e-854a-d1e35c2c1824 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:17.371792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad2689-7950-431c-88bb-213d03027f5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:17.381056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-edad2689-7950-431c-88bb-213d03027f5c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:58:17.381056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:17.381056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:17.381056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:17.381056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:17.381056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:17.381056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:17.381056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:17.381056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:17.381056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:17.381056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:17.381056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:17.381056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:17.381056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:17.381056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:17.381056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:17.381964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:17.381964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:17.381964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:17.381964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:17.381964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:17.381964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:17.381964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:17.381964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:17.381964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:17.381964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:17.381964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:17.381964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:17.381964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:17.381964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:17.381964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:17.381964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:17.383023 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:17.383023 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:17.383023 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:17.383023 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:17.383023 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:17.383023 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:17.383023 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:17.383023 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:17.383023 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:17.383023 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:17.383023 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:58:17.383023 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:17.385835 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 194/386] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 15:58:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:17.392628 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad2689-7950-431c-88bb-213d03027f5c None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:17.437098 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adbe292-0172-4559-9605-8d559cdc133d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:17.438176 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adbe292-0172-4559-9605-8d559cdc133d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:17.439813 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adbe292-0172-4559-9605-8d559cdc133d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:17.440300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adbe292-0172-4559-9605-8d559cdc133d None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:17.441475 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adbe292-0172-4559-9605-8d559cdc133d None admin] RBAC: Adding query filter params (name=member) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:17.443123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adbe292-0172-4559-9605-8d559cdc133d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:17.446225 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adbe292-0172-4559-9605-8d559cdc133d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:17.461998 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 193/387] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 15:58:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:17.472101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adbe292-0172-4559-9605-8d559cdc133d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:17.577648 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:17.593371 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad2689-7950-431c-88bb-213d03027f5c None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:17.666549 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650879f-b857-4160-8405-214e5ba7c8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:17.667141 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650879f-b857-4160-8405-214e5ba7c8b5 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:17.667420 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650879f-b857-4160-8405-214e5ba7c8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:17.667693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650879f-b857-4160-8405-214e5ba7c8b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:17.668442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1650879f-b857-4160-8405-214e5ba7c8b5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:17.668747 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1650879f-b857-4160-8405-214e5ba7c8b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:17.674429 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adbe292-0172-4559-9605-8d559cdc133d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:17.676319 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1650879f-b857-4160-8405-214e5ba7c8b5 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:17.695487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 195/388] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:17.699686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650879f-b857-4160-8405-214e5ba7c8b5 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:17.743350 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bbae96-2532-4af4-b042-e65d9bc96f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:17.744222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bbae96-2532-4af4-b042-e65d9bc96f87 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:17.744504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bbae96-2532-4af4-b042-e65d9bc96f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:17.744776 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bbae96-2532-4af4-b042-e65d9bc96f87 None admin] PATH_INFO: `/v3/groups/ee669194886e4a1c9d35bf7d4fb62d91` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:17.747329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bbae96-2532-4af4-b042-e65d9bc96f87 None admin] RBAC: Authorizing `identity:get_group(group_id=ee669194886e4a1c9d35bf7d4fb62d91)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:17.749732 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bbae96-2532-4af4-b042-e65d9bc96f87 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:17.754174 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 194/389] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 15:58:17 2022] GET /identity/v3/groups/ee669194886e4a1c9d35bf7d4fb62d91 => generated 226 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:17.757821 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bbae96-2532-4af4-b042-e65d9bc96f87 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:17.827707 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:17.891041 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650879f-b857-4160-8405-214e5ba7c8b5 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:17.933348 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b3eb1-c08c-47cf-8534-e813021fb79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:17.935853 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b3eb1-c08c-47cf-8534-e813021fb79e None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:17.936521 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b3eb1-c08c-47cf-8534-e813021fb79e None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:17.938681 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b3eb1-c08c-47cf-8534-e813021fb79e None admin] PATH_INFO: `/v3/groups/ee669194886e4a1c9d35bf7d4fb62d91` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:17.941833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b3eb1-c08c-47cf-8534-e813021fb79e None admin] RBAC: Authorizing `identity:get_group(group_id=ee669194886e4a1c9d35bf7d4fb62d91)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:17.943724 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b3eb1-c08c-47cf-8534-e813021fb79e None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:17.947771 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 196/390] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 15:58:17 2022] GET /identity/v3/groups/ee669194886e4a1c9d35bf7d4fb62d91 => generated 226 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:17.956977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b3eb1-c08c-47cf-8534-e813021fb79e None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:17.973485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bbae96-2532-4af4-b042-e65d9bc96f87 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:17.982646 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:18.025594 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff651d77-7dea-462c-bf73-d93d43e01f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:18.026492 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff651d77-7dea-462c-bf73-d93d43e01f6b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:18.026820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff651d77-7dea-462c-bf73-d93d43e01f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:18.027103 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff651d77-7dea-462c-bf73-d93d43e01f6b None admin] PATH_INFO: `/v3/projects/a83dc386fda8411eb2d4aef8e43b7065` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:18.028283 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff651d77-7dea-462c-bf73-d93d43e01f6b None admin] RBAC: Authorizing `identity:get_project(project_id=a83dc386fda8411eb2d4aef8e43b7065)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:18.029946 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff651d77-7dea-462c-bf73-d93d43e01f6b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:18.031996 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 195/391] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:17 2022] GET /identity/v3/projects/a83dc386fda8411eb2d4aef8e43b7065 => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:18.033291 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216f164-d99b-4cd9-b682-3bb1a0626770 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:18.033523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216f164-d99b-4cd9-b682-3bb1a0626770 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:18.033803 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216f164-d99b-4cd9-b682-3bb1a0626770 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:18.039157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 196/392] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:18.039879 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216f164-d99b-4cd9-b682-3bb1a0626770 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:18.084641 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b3eb1-c08c-47cf-8534-e813021fb79e None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:18.130261 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02c9fe-979b-4d7c-a5b9-3085639c0bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:18.132051 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02c9fe-979b-4d7c-a5b9-3085639c0bec None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:18.133093 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02c9fe-979b-4d7c-a5b9-3085639c0bec None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:18.133581 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02c9fe-979b-4d7c-a5b9-3085639c0bec None admin] PATH_INFO: `/v3/projects/a83dc386fda8411eb2d4aef8e43b7065` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:18.135211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f02c9fe-979b-4d7c-a5b9-3085639c0bec None admin] RBAC: Authorizing `identity:get_project(project_id=a83dc386fda8411eb2d4aef8e43b7065)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:18.137011 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f02c9fe-979b-4d7c-a5b9-3085639c0bec None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:18.144733 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 197/393] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:17 2022] GET /identity/v3/projects/a83dc386fda8411eb2d4aef8e43b7065 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:18.146697 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bd958-2600-4e27-9d4a-18ff008514b7 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:18.147239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bd958-2600-4e27-9d4a-18ff008514b7 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:18.148030 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bd958-2600-4e27-9d4a-18ff008514b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:18.172589 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-932bd958-2600-4e27-9d4a-18ff008514b7 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:18.175034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-932bd958-2600-4e27-9d4a-18ff008514b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:18.183466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:18.191637 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216f164-d99b-4cd9-b682-3bb1a0626770 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:18.252533 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5dad0-98de-4827-b965-6909f78e5101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:18.253781 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5dad0-98de-4827-b965-6909f78e5101 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:18.254552 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5dad0-98de-4827-b965-6909f78e5101 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:18.255016 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5dad0-98de-4827-b965-6909f78e5101 None admin] PATH_INFO: `/v3/projects/a83dc386fda8411eb2d4aef8e43b7065/groups/ee669194886e4a1c9d35bf7d4fb62d91/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:18.258256 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5dad0-98de-4827-b965-6909f78e5101 None admin] RBAC: Authorizing `identity:create_grant(project_id=a83dc386fda8411eb2d4aef8e43b7065, group_id=ee669194886e4a1c9d35bf7d4fb62d91, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:18.260613 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5dad0-98de-4827-b965-6909f78e5101 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:18.280481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 197/395] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jun 16 15:58:18 2022] PUT /identity/v3/projects/a83dc386fda8411eb2d4aef8e43b7065/groups/ee669194886e4a1c9d35bf7d4fb62d91/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:58:18.281373 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 198/395] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:18.281823 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5dad0-98de-4827-b965-6909f78e5101 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:18.302666 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932bd958-2600-4e27-9d4a-18ff008514b7 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:18.325169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:18.450482 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932bd958-2600-4e27-9d4a-18ff008514b7 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:18.470412 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5dad0-98de-4827-b965-6909f78e5101 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:18.499505 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d841b-9672-4677-b205-1fad1dbd4548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:18.501973 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d841b-9672-4677-b205-1fad1dbd4548 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:18.501973 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d841b-9672-4677-b205-1fad1dbd4548 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:18.501973 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d841b-9672-4677-b205-1fad1dbd4548 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:18.514854 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d841b-9672-4677-b205-1fad1dbd4548 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:18.525273 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ddeeef-3109-449a-afce-51666874162c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:18.525652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-893d841b-9672-4677-b205-1fad1dbd4548 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:18.525652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:18.525652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:18.525652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:18.525652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:18.525652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:18.525652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:18.525652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:18.525652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:18.525652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:18.525652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:18.525652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:18.525652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:18.525652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:18.525652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:18.525652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:18.526618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:18.526618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:18.526618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:18.526618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:18.526618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:18.526618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:18.526618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:18.526618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:18.526618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:18.526618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:18.526618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:18.526618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:18.526618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:18.526618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:18.526618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:18.526618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:18.532440 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:18.532440 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:18.532440 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:18.532440 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:18.532440 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:18.532440 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:18.532440 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:18.532440 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:18.532440 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:18.532440 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:18.532440 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:18.532440 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:18.532440 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 199/396] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:58:18.532440 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a28046-cdbe-43f6-8dda-6b59733d11db None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:18.533420 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a28046-cdbe-43f6-8dda-6b59733d11db None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:18.533420 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a28046-cdbe-43f6-8dda-6b59733d11db None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:18.533420 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 200/397] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:18.534679 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddeeef-3109-449a-afce-51666874162c None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:18.535093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddeeef-3109-449a-afce-51666874162c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:18.535571 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddeeef-3109-449a-afce-51666874162c None admin] PATH_INFO: `/v3/projects/a83dc386fda8411eb2d4aef8e43b7065/groups/ee669194886e4a1c9d35bf7d4fb62d91/roles/e0b0c216ef51441ab34b6e036da09daa` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:18.537116 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eeca95-edb7-4693-b07c-889f67b0469d None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:18.537411 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eeca95-edb7-4693-b07c-889f67b0469d None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:18.537690 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eeca95-edb7-4693-b07c-889f67b0469d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:18.538617 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ddeeef-3109-449a-afce-51666874162c None admin] RBAC: Authorizing `identity:create_grant(project_id=a83dc386fda8411eb2d4aef8e43b7065, group_id=ee669194886e4a1c9d35bf7d4fb62d91, role_id=e0b0c216ef51441ab34b6e036da09daa)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:18.540513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ddeeef-3109-449a-afce-51666874162c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:18.563171 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-30eeca95-edb7-4693-b07c-889f67b0469d None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:18.565302 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-30eeca95-edb7-4693-b07c-889f67b0469d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:18.566229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 198/398] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jun 16 15:58:18 2022] PUT /identity/v3/projects/a83dc386fda8411eb2d4aef8e43b7065/groups/ee669194886e4a1c9d35bf7d4fb62d91/roles/e0b0c216ef51441ab34b6e036da09daa => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:58:18.568533 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ddeeef-3109-449a-afce-51666874162c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:18.614372 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:18.665686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 201/399] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:58:18.689652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eeca95-edb7-4693-b07c-889f67b0469d None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:18.704793 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:18.740037 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ddeeef-3109-449a-afce-51666874162c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:18.843388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eeca95-edb7-4693-b07c-889f67b0469d None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:18.854746 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26707482-649a-4970-929b-98138802ea3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:18.855394 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26707482-649a-4970-929b-98138802ea3e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:18.855662 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26707482-649a-4970-929b-98138802ea3e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:18.855938 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26707482-649a-4970-929b-98138802ea3e None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:18.856706 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26707482-649a-4970-929b-98138802ea3e None admin] RBAC: Adding query filter params (name=member) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:18.857013 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26707482-649a-4970-929b-98138802ea3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:18.859144 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26707482-649a-4970-929b-98138802ea3e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:18.869381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 199/400] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 15:58:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:18.876021 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26707482-649a-4970-929b-98138802ea3e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:18.898426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59deb25a-0a88-4f15-8b8c-694eb6be34af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:18.899506 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59deb25a-0a88-4f15-8b8c-694eb6be34af None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:18.899795 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59deb25a-0a88-4f15-8b8c-694eb6be34af None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:18.900078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59deb25a-0a88-4f15-8b8c-694eb6be34af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:18.914007 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59deb25a-0a88-4f15-8b8c-694eb6be34af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:18.923239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-59deb25a-0a88-4f15-8b8c-694eb6be34af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:58:18.923239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:18.923239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:18.923239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:18.923239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:18.923239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:18.923239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:18.923239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:18.923239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:18.923239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:18.923239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:18.923239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:18.923239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:18.923239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:18.923239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:18.923239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:18.924342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:18.924342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:18.924342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:18.924342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:18.924342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:18.924342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:18.924342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:18.924342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:18.924342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:18.924342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:18.924342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:18.924342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:18.924342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:18.924342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:18.924342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:18.924342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:18.925975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:18.925975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:18.925975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:18.925975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:18.925975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:18.925975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:18.925975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:18.925975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:18.925975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:18.925975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:18.925975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:58:18.925975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:18.925975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 202/401] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 15:58:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:58:18.925975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427bf54-5ede-45b7-8987-2773ee4a3450 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:18.926896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427bf54-5ede-45b7-8987-2773ee4a3450 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:18.926896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427bf54-5ede-45b7-8987-2773ee4a3450 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:18.928096 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 203/402] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:18.933562 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427bf54-5ede-45b7-8987-2773ee4a3450 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:19.012027 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:19.026895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:19.052111 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26707482-649a-4970-929b-98138802ea3e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:19.073244 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427bf54-5ede-45b7-8987-2773ee4a3450 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:19.090848 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad1021-7a27-4dce-ac0f-c1009d4a66f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:19.093167 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad1021-7a27-4dce-ac0f-c1009d4a66f4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:19.093704 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad1021-7a27-4dce-ac0f-c1009d4a66f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:19.094175 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad1021-7a27-4dce-ac0f-c1009d4a66f4 None admin] PATH_INFO: `/v3/groups/ee669194886e4a1c9d35bf7d4fb62d91` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:19.095653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ad1021-7a27-4dce-ac0f-c1009d4a66f4 None admin] RBAC: Authorizing `identity:get_group(group_id=ee669194886e4a1c9d35bf7d4fb62d91)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:19.098179 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ad1021-7a27-4dce-ac0f-c1009d4a66f4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:19.100697 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 200/403] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 15:58:18 2022] GET /identity/v3/groups/ee669194886e4a1c9d35bf7d4fb62d91 => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:19.102309 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11c830-c366-4038-8a07-a17ddb6902dc None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:19.102815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11c830-c366-4038-8a07-a17ddb6902dc None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:19.103384 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11c830-c366-4038-8a07-a17ddb6902dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:19.117582 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf64d55-d6c2-4ba1-9aed-a30965a38312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:19.118081 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf64d55-d6c2-4ba1-9aed-a30965a38312 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:19.118364 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf64d55-d6c2-4ba1-9aed-a30965a38312 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:19.118638 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf64d55-d6c2-4ba1-9aed-a30965a38312 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:19.119501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf64d55-d6c2-4ba1-9aed-a30965a38312 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:19.119908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf64d55-d6c2-4ba1-9aed-a30965a38312 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:19.121285 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf64d55-d6c2-4ba1-9aed-a30965a38312 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:19.124716 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0e11c830-c366-4038-8a07-a17ddb6902dc None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:19.126191 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0e11c830-c366-4038-8a07-a17ddb6902dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:19.132741 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 204/404] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:19.133703 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf64d55-d6c2-4ba1-9aed-a30965a38312 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:19.209573 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 201/405] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:19.211126 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e11c830-c366-4038-8a07-a17ddb6902dc None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:19.275201 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf64d55-d6c2-4ba1-9aed-a30965a38312 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:19.309872 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:19.320335 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b122ffc-81b6-422b-a581-7bf869941a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:19.321433 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b122ffc-81b6-422b-a581-7bf869941a03 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:19.321721 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b122ffc-81b6-422b-a581-7bf869941a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:19.322058 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b122ffc-81b6-422b-a581-7bf869941a03 None admin] PATH_INFO: `/v3/projects/328b693aa62e4467aa34a749006ad00f` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:19.323987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b122ffc-81b6-422b-a581-7bf869941a03 None admin] RBAC: Authorizing `identity:get_project(project_id=328b693aa62e4467aa34a749006ad00f)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:19.326892 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b122ffc-81b6-422b-a581-7bf869941a03 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:19.328829 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 205/406] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:19 2022] GET /identity/v3/projects/328b693aa62e4467aa34a749006ad00f => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:19.329805 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b122ffc-81b6-422b-a581-7bf869941a03 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:19.336521 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:19.372100 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e11c830-c366-4038-8a07-a17ddb6902dc None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:19.429338 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac64d6-5d5c-4699-aee4-0007616208fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:19.431235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac64d6-5d5c-4699-aee4-0007616208fa None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:19.431724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac64d6-5d5c-4699-aee4-0007616208fa None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:19.432245 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac64d6-5d5c-4699-aee4-0007616208fa None admin] PATH_INFO: `/v3/groups/ee669194886e4a1c9d35bf7d4fb62d91` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:19.434581 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac64d6-5d5c-4699-aee4-0007616208fa None admin] RBAC: Authorizing `identity:get_group(group_id=ee669194886e4a1c9d35bf7d4fb62d91)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:19.438516 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac64d6-5d5c-4699-aee4-0007616208fa None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:19.441115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 202/407] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 15:58:19 2022] GET /identity/v3/groups/ee669194886e4a1c9d35bf7d4fb62d91 => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:19.442796 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac64d6-5d5c-4699-aee4-0007616208fa None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:19.463772 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b122ffc-81b6-422b-a581-7bf869941a03 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:19.515565 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aec152-e175-4db6-8dee-c04773514c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:19.516610 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aec152-e175-4db6-8dee-c04773514c7b None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:19.517058 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aec152-e175-4db6-8dee-c04773514c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:19.517642 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aec152-e175-4db6-8dee-c04773514c7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:19.532842 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aec152-e175-4db6-8dee-c04773514c7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:19.542020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-70aec152-e175-4db6-8dee-c04773514c7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:19.542020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:19.542020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:19.542020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:19.542020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:19.542020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:19.542020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:19.542020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:19.542020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:19.542020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:19.542020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:19.542020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:19.542020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:19.542020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:19.542020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:19.542020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:19.543126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:19.543126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:19.543126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:19.543126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:19.543126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:19.543126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:19.543126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:19.543126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:19.543126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:19.543126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:19.543126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:19.543126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:19.543126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:19.543126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:19.543126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:19.543126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:19.544087 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:19.544087 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:19.544087 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:19.544087 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:19.544087 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:19.544087 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:19.544087 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:19.544087 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:19.544087 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:19.544087 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:19.544087 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:19.544087 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:19.546587 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 206/408] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 15:58:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:19.548110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aec152-e175-4db6-8dee-c04773514c7b None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:19.563225 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac64d6-5d5c-4699-aee4-0007616208fa None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:19.622622 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eebc98-49ba-4cc4-88b3-319a88ed821b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:19.625250 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eebc98-49ba-4cc4-88b3-319a88ed821b None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:19.625711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eebc98-49ba-4cc4-88b3-319a88ed821b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:19.626153 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eebc98-49ba-4cc4-88b3-319a88ed821b None admin] PATH_INFO: `/v3/projects/328b693aa62e4467aa34a749006ad00f/groups/ee669194886e4a1c9d35bf7d4fb62d91/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:19.626892 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:19.630593 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eebc98-49ba-4cc4-88b3-319a88ed821b None admin] RBAC: Authorizing `identity:create_grant(project_id=328b693aa62e4467aa34a749006ad00f, group_id=ee669194886e4a1c9d35bf7d4fb62d91, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:19.633412 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eebc98-49ba-4cc4-88b3-319a88ed821b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:19.657610 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 203/409] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jun 16 15:58:19 2022] PUT /identity/v3/projects/328b693aa62e4467aa34a749006ad00f/groups/ee669194886e4a1c9d35bf7d4fb62d91/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:58:19.658739 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eebc98-49ba-4cc4-88b3-319a88ed821b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:19.673755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:19.699520 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aec152-e175-4db6-8dee-c04773514c7b None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:19.774050 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b8207-03d0-4fa5-b536-5bdf436fed56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:19.776438 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b8207-03d0-4fa5-b536-5bdf436fed56 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:19.777098 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b8207-03d0-4fa5-b536-5bdf436fed56 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:19.777856 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b8207-03d0-4fa5-b536-5bdf436fed56 None admin] PATH_INFO: `/v3/projects/328b693aa62e4467aa34a749006ad00f` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:19.780210 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b8207-03d0-4fa5-b536-5bdf436fed56 None admin] RBAC: Authorizing `identity:get_project(project_id=328b693aa62e4467aa34a749006ad00f)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:19.782227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b8207-03d0-4fa5-b536-5bdf436fed56 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:19.784884 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 207/410] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:19 2022] GET /identity/v3/projects/328b693aa62e4467aa34a749006ad00f => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:19.792747 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b8207-03d0-4fa5-b536-5bdf436fed56 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:19.821726 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eebc98-49ba-4cc4-88b3-319a88ed821b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:19.913629 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65bb32e-d650-45fc-95bd-ae88a5404034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:19.915393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65bb32e-d650-45fc-95bd-ae88a5404034 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:19.917176 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65bb32e-d650-45fc-95bd-ae88a5404034 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:19.919157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65bb32e-d650-45fc-95bd-ae88a5404034 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:19.920815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65bb32e-d650-45fc-95bd-ae88a5404034 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:19.921792 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65bb32e-d650-45fc-95bd-ae88a5404034 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:19.927628 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65bb32e-d650-45fc-95bd-ae88a5404034 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:19.942574 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 204/411] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:19.951814 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65bb32e-d650-45fc-95bd-ae88a5404034 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:19.995356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b8207-03d0-4fa5-b536-5bdf436fed56 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:20.056797 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ea2fb-011d-4542-9fd0-7583b5abfe9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:20.057704 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ea2fb-011d-4542-9fd0-7583b5abfe9b None admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:20.058013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ea2fb-011d-4542-9fd0-7583b5abfe9b None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:20.058290 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ea2fb-011d-4542-9fd0-7583b5abfe9b None admin] PATH_INFO: `/v3/projects/328b693aa62e4467aa34a749006ad00f/groups/ee669194886e4a1c9d35bf7d4fb62d91/roles/e0b0c216ef51441ab34b6e036da09daa` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:20.062371 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:20.067195 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ea2fb-011d-4542-9fd0-7583b5abfe9b None admin] RBAC: Authorizing `identity:create_grant(project_id=328b693aa62e4467aa34a749006ad00f, group_id=ee669194886e4a1c9d35bf7d4fb62d91, role_id=e0b0c216ef51441ab34b6e036da09daa)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:20.072437 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ea2fb-011d-4542-9fd0-7583b5abfe9b None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:20.104528 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 208/412] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jun 16 15:58:19 2022] PUT /identity/v3/projects/328b693aa62e4467aa34a749006ad00f/groups/ee669194886e4a1c9d35bf7d4fb62d91/roles/e0b0c216ef51441ab34b6e036da09daa => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:58:20.112817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:20.198886 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65bb32e-d650-45fc-95bd-ae88a5404034 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:20.265734 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f661f-12ed-4d1f-9c50-8a8cb60bf725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:20.267334 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f661f-12ed-4d1f-9c50-8a8cb60bf725 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:20.267833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f661f-12ed-4d1f-9c50-8a8cb60bf725 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:20.268329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f661f-12ed-4d1f-9c50-8a8cb60bf725 None admin] PATH_INFO: `/v3/groups/15b9a6ed0fdb440e93c60902d1603904` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:20.270506 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f661f-12ed-4d1f-9c50-8a8cb60bf725 None admin] RBAC: Authorizing `identity:get_group(group_id=15b9a6ed0fdb440e93c60902d1603904)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:20.273290 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f661f-12ed-4d1f-9c50-8a8cb60bf725 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:20.277033 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 205/413] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 15:58:19 2022] GET /identity/v3/groups/15b9a6ed0fdb440e93c60902d1603904 => generated 229 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:20.283266 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ea2fb-011d-4542-9fd0-7583b5abfe9b None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:20.425789 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ea2fb-011d-4542-9fd0-7583b5abfe9b None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:20.465327 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d050e-e172-4ed0-81d0-7a150d04e11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:20.466237 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d050e-e172-4ed0-81d0-7a150d04e11b None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:20.466558 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d050e-e172-4ed0-81d0-7a150d04e11b None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:20.466842 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d050e-e172-4ed0-81d0-7a150d04e11b None admin] PATH_INFO: `/v3/projects/47ead53fd5cc424c8de9383e832d40a2` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:20.468121 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d050e-e172-4ed0-81d0-7a150d04e11b None admin] RBAC: Authorizing `identity:get_project(project_id=47ead53fd5cc424c8de9383e832d40a2)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:20.469613 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d050e-e172-4ed0-81d0-7a150d04e11b None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:20.471705 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 209/414] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:20 2022] GET /identity/v3/projects/47ead53fd5cc424c8de9383e832d40a2 => generated 344 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:20.477611 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f661f-12ed-4d1f-9c50-8a8cb60bf725 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:20.623881 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f661f-12ed-4d1f-9c50-8a8cb60bf725 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:20.680349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc8d1f9-145e-44fa-97ea-dce2873a5be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:20.681177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8d1f9-145e-44fa-97ea-dce2873a5be9 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:20.681468 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8d1f9-145e-44fa-97ea-dce2873a5be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:20.681745 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8d1f9-145e-44fa-97ea-dce2873a5be9 None admin] PATH_INFO: `/v3/projects/47ead53fd5cc424c8de9383e832d40a2/groups/15b9a6ed0fdb440e93c60902d1603904/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:20.687407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc8d1f9-145e-44fa-97ea-dce2873a5be9 None admin] RBAC: Authorizing `identity:create_grant(project_id=47ead53fd5cc424c8de9383e832d40a2, group_id=15b9a6ed0fdb440e93c60902d1603904, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:20.688905 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc8d1f9-145e-44fa-97ea-dce2873a5be9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:20.695213 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:20.710789 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 206/415] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jun 16 15:58:20 2022] PUT /identity/v3/projects/47ead53fd5cc424c8de9383e832d40a2/groups/15b9a6ed0fdb440e93c60902d1603904/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:58:21.318053 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc9e12-698c-46f4-addd-bc445fe918c2 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:21.319097 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc9e12-698c-46f4-addd-bc445fe918c2 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:21.319613 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc9e12-698c-46f4-addd-bc445fe918c2 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:21.321802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 210/416] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:21.327930 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a8cf5-922c-4cd7-b335-4f7b52d9e705 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:21.328864 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a8cf5-922c-4cd7-b335-4f7b52d9e705 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:21.329430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a8cf5-922c-4cd7-b335-4f7b52d9e705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:21.338704 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f353bb5-1521-4b1b-b318-ba115c9d4438 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:21.339336 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f353bb5-1521-4b1b-b318-ba115c9d4438 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:21.339816 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f353bb5-1521-4b1b-b318-ba115c9d4438 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:21.341426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 211/417] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:58:21.354524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a5edd-b398-466f-918b-a4d9ee133dad None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:21.355126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a5edd-b398-466f-918b-a4d9ee133dad None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:21.355607 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a5edd-b398-466f-918b-a4d9ee133dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:21.362262 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b07a8cf5-922c-4cd7-b335-4f7b52d9e705 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:21.363638 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b07a8cf5-922c-4cd7-b335-4f7b52d9e705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:21.380426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-d18a5edd-b398-466f-918b-a4d9ee133dad None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:21.380426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-d18a5edd-b398-466f-918b-a4d9ee133dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:21.501589 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:21.513965 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 207/418] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:58:21.528347 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 212/419] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:21.546615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a8cf5-922c-4cd7-b335-4f7b52d9e705 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:21.565213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a5edd-b398-466f-918b-a4d9ee133dad None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:21.695287 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a8cf5-922c-4cd7-b335-4f7b52d9e705 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:21.725571 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:21.729619 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a5edd-b398-466f-918b-a4d9ee133dad None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:21.747788 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236d651-f6be-4659-b810-27e9a7e3f656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:21.748659 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236d651-f6be-4659-b810-27e9a7e3f656 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:21.748988 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236d651-f6be-4659-b810-27e9a7e3f656 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:21.749305 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236d651-f6be-4659-b810-27e9a7e3f656 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:21.774130 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4236d651-f6be-4659-b810-27e9a7e3f656 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:21.789725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-4236d651-f6be-4659-b810-27e9a7e3f656 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:58:21.789725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:21.789725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:21.789725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:21.789725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:21.789725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:21.789725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:21.789725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:21.789725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:21.789725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:21.789725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:21.789725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:21.789725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:21.789725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:21.789725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:21.789725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:21.790869 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:21.790869 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:21.790869 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:21.790869 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:21.790869 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:21.790869 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:21.790869 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:21.790869 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:21.790869 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:21.790869 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:21.790869 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:21.790869 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:21.790869 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:21.790869 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:21.790869 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:21.790869 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:21.792236 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:21.792236 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:21.792236 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:21.792236 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:21.792236 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:21.792236 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:21.792236 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:21.792236 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:21.792236 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:21.792236 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:21.792236 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:58:21.792236 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:21.796182 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58a1dc-ceab-41a2-ac79-78be4d129a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:21.797288 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58a1dc-ceab-41a2-ac79-78be4d129a03 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:21.797791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58a1dc-ceab-41a2-ac79-78be4d129a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:21.798310 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58a1dc-ceab-41a2-ac79-78be4d129a03 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:21.802102 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 208/420] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 15:58:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:21.807760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236d651-f6be-4659-b810-27e9a7e3f656 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:21.817823 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd58a1dc-ceab-41a2-ac79-78be4d129a03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:21.831564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-dd58a1dc-ceab-41a2-ac79-78be4d129a03 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:21.831564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:21.831564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:21.831564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:21.831564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:21.831564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:21.831564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:21.831564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:21.831564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:21.831564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:21.831564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:21.831564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:21.831564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:21.831564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:21.831564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:21.831564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:21.832529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:21.832529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:21.832529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:21.832529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:21.832529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:21.832529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:21.832529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:21.832529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:21.832529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:21.832529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:21.832529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:21.832529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:21.832529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:21.832529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:21.832529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:21.832529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:21.833340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:21.833340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:21.833340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:21.833340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:21.833340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:21.833340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:21.833340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:21.833340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:21.833340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:21.833340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:21.833340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:21.833340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:21.840533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 213/421] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:58:21.846429 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58a1dc-ceab-41a2-ac79-78be4d129a03 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:21.902332 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:21.963358 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58a1dc-ceab-41a2-ac79-78be4d129a03 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:22.004410 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ee800-813e-48b6-819a-dc41ab70b774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:22.005570 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ee800-813e-48b6-819a-dc41ab70b774 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:22.006205 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ee800-813e-48b6-819a-dc41ab70b774 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:22.006717 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ee800-813e-48b6-819a-dc41ab70b774 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:22.007958 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ee800-813e-48b6-819a-dc41ab70b774 None admin] RBAC: Adding query filter params (name=member) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:22.008487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ee800-813e-48b6-819a-dc41ab70b774 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:22.010158 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ee800-813e-48b6-819a-dc41ab70b774 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:22.021110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 214/422] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 15:58:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:22.026742 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236d651-f6be-4659-b810-27e9a7e3f656 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:22.033055 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ee800-813e-48b6-819a-dc41ab70b774 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:22.087947 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c41c0-a2e4-40d2-a319-e65e5e3d7623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:22.088638 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c41c0-a2e4-40d2-a319-e65e5e3d7623 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:22.088918 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c41c0-a2e4-40d2-a319-e65e5e3d7623 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:22.089193 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c41c0-a2e4-40d2-a319-e65e5e3d7623 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:22.090956 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c41c0-a2e4-40d2-a319-e65e5e3d7623 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:22.091276 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c41c0-a2e4-40d2-a319-e65e5e3d7623 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:22.093658 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c41c0-a2e4-40d2-a319-e65e5e3d7623 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:22.116539 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 209/423] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:22.125422 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c41c0-a2e4-40d2-a319-e65e5e3d7623 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:22.202740 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ee800-813e-48b6-819a-dc41ab70b774 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:22.210290 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:22.215657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:22.241623 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918dc30-0ef6-4d2c-b544-23d1e3bbaf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:22.242479 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918dc30-0ef6-4d2c-b544-23d1e3bbaf39 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:22.242633 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918dc30-0ef6-4d2c-b544-23d1e3bbaf39 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:22.243077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918dc30-0ef6-4d2c-b544-23d1e3bbaf39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:22.319126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3918dc30-0ef6-4d2c-b544-23d1e3bbaf39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:22.321443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3918dc30-0ef6-4d2c-b544-23d1e3bbaf39 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:22.361760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c41c0-a2e4-40d2-a319-e65e5e3d7623 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:22.427175 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652de8fe-eb2d-445e-affe-34acb3f08784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:22.428246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652de8fe-eb2d-445e-affe-34acb3f08784 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:22.428702 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652de8fe-eb2d-445e-affe-34acb3f08784 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:22.429122 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652de8fe-eb2d-445e-affe-34acb3f08784 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:22.443373 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 215/424] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 15:58:22.445160 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33338548-f656-4fb0-9220-1a204026371f None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:22.445601 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33338548-f656-4fb0-9220-1a204026371f None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:22.446080 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33338548-f656-4fb0-9220-1a204026371f None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:22.448081 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 216/425] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:22.448955 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33338548-f656-4fb0-9220-1a204026371f None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:22.458949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:22.524871 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652de8fe-eb2d-445e-affe-34acb3f08784 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:22.527102 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652de8fe-eb2d-445e-affe-34acb3f08784 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:22.593484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33338548-f656-4fb0-9220-1a204026371f None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:22.630461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:22.638448 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30a21f-52be-47ce-8730-c3a299831db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:22.640103 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30a21f-52be-47ce-8730-c3a299831db4 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:22.640533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30a21f-52be-47ce-8730-c3a299831db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:22.640970 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30a21f-52be-47ce-8730-c3a299831db4 None admin] PATH_INFO: `/v3/projects/a83dc386fda8411eb2d4aef8e43b7065` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:22.643326 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b30a21f-52be-47ce-8730-c3a299831db4 None admin] RBAC: Authorizing `identity:get_project(project_id=a83dc386fda8411eb2d4aef8e43b7065)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:22.645494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b30a21f-52be-47ce-8730-c3a299831db4 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:22.648228 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 217/426] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:22 2022] GET /identity/v3/projects/a83dc386fda8411eb2d4aef8e43b7065 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:22.649660 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d236fd3-b79f-4e2b-bede-c2ccf8d8ef4d None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:22.650129 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d236fd3-b79f-4e2b-bede-c2ccf8d8ef4d None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:22.650537 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d236fd3-b79f-4e2b-bede-c2ccf8d8ef4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:22.667210 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 210/427] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 15:58:22.668331 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652de8fe-eb2d-445e-affe-34acb3f08784 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:22.677278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-0d236fd3-b79f-4e2b-bede-c2ccf8d8ef4d None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:22.679324 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-0d236fd3-b79f-4e2b-bede-c2ccf8d8ef4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:22.770555 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 218/428] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:22.771710 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d236fd3-b79f-4e2b-bede-c2ccf8d8ef4d None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:22.780693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:22.815718 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652de8fe-eb2d-445e-affe-34acb3f08784 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:22.855168 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397efdc-50a4-4b68-beff-e488327cc93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:22.856640 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397efdc-50a4-4b68-beff-e488327cc93e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:22.857131 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397efdc-50a4-4b68-beff-e488327cc93e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:22.857876 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397efdc-50a4-4b68-beff-e488327cc93e None admin] PATH_INFO: `/v3/groups/ee669194886e4a1c9d35bf7d4fb62d91` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:22.859835 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b397efdc-50a4-4b68-beff-e488327cc93e None admin] RBAC: Authorizing `identity:get_group(group_id=ee669194886e4a1c9d35bf7d4fb62d91)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:22.861699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b397efdc-50a4-4b68-beff-e488327cc93e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:22.869205 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 211/429] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 15:58:22 2022] GET /identity/v3/groups/ee669194886e4a1c9d35bf7d4fb62d91 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:22.870431 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397efdc-50a4-4b68-beff-e488327cc93e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:22.892561 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:22.911116 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d236fd3-b79f-4e2b-bede-c2ccf8d8ef4d None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:22.957342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4657cc-5161-4f45-b34b-33d5bd410b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:22.958244 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4657cc-5161-4f45-b34b-33d5bd410b73 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:22.958530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4657cc-5161-4f45-b34b-33d5bd410b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:22.958810 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4657cc-5161-4f45-b34b-33d5bd410b73 None admin] PATH_INFO: `/v3/projects/a83dc386fda8411eb2d4aef8e43b7065` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:22.960052 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4657cc-5161-4f45-b34b-33d5bd410b73 None admin] RBAC: Authorizing `identity:get_project(project_id=a83dc386fda8411eb2d4aef8e43b7065)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:22.962056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4657cc-5161-4f45-b34b-33d5bd410b73 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:22.968840 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 219/430] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:22 2022] GET /identity/v3/projects/a83dc386fda8411eb2d4aef8e43b7065 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:22.968840 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4657cc-5161-4f45-b34b-33d5bd410b73 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:22.995555 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397efdc-50a4-4b68-beff-e488327cc93e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:23.044489 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c0e9b-d970-4e78-921b-9de91a261b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:23.046116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c0e9b-d970-4e78-921b-9de91a261b01 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:23.046618 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c0e9b-d970-4e78-921b-9de91a261b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:23.047095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c0e9b-d970-4e78-921b-9de91a261b01 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:23.055322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:23.058981 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c0e9b-d970-4e78-921b-9de91a261b01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:23.068059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-c46c0e9b-d970-4e78-921b-9de91a261b01 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:23.068059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:23.068059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:23.068059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:23.068059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:23.068059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:23.068059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:23.068059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:23.068059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:23.068059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:23.068059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:23.068059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:23.068059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:23.068059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:23.068059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:23.068059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:23.069218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:23.069218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:23.069218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:23.069218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:23.069218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:23.069218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:23.069218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:23.069218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:23.069218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:23.069218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:23.069218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:23.069218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:23.069218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:23.069218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:23.069218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:23.069218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:23.070391 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:23.070391 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:23.070391 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:23.070391 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:23.070391 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:23.070391 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:23.070391 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:23.070391 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:23.070391 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:23.070391 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:23.070391 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:23.070391 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:23.071437 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 212/431] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:23.072246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c0e9b-d970-4e78-921b-9de91a261b01 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:23.093964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4657cc-5161-4f45-b34b-33d5bd410b73 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:23.143339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2263ed-5696-47a2-8b1d-4fc7c53120a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:23.144648 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2263ed-5696-47a2-8b1d-4fc7c53120a2 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:23.145164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2263ed-5696-47a2-8b1d-4fc7c53120a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:23.145670 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2263ed-5696-47a2-8b1d-4fc7c53120a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:23.147344 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2263ed-5696-47a2-8b1d-4fc7c53120a2 None admin] RBAC: Adding query filter params (group.id=ee669194886e4a1c9d35bf7d4fb62d91, role.id=3ce570490865493380e6941d004136bd, scope.project.id=a83dc386fda8411eb2d4aef8e43b7065) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:23.147929 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2263ed-5696-47a2-8b1d-4fc7c53120a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:23.150750 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2263ed-5696-47a2-8b1d-4fc7c53120a2 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:23.160683 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 220/432] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jun 16 15:58:22 2022] GET /identity/v3/role_assignments?group.id=ee669194886e4a1c9d35bf7d4fb62d91&scope.project.id=a83dc386fda8411eb2d4aef8e43b7065&role.id=3ce570490865493380e6941d004136bd => generated 611 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:23.162104 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb9e3ee-0b77-4037-9b8a-67999a503960 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:23.162432 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb9e3ee-0b77-4037-9b8a-67999a503960 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:23.162710 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb9e3ee-0b77-4037-9b8a-67999a503960 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:23.168105 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 221/433] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:23.169035 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb9e3ee-0b77-4037-9b8a-67999a503960 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:23.170461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:23.213901 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c0e9b-d970-4e78-921b-9de91a261b01 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:23.259052 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb42039-a378-4711-a34c-3f3f4a7a29c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:23.262146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb42039-a378-4711-a34c-3f3f4a7a29c2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:23.262839 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb42039-a378-4711-a34c-3f3f4a7a29c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:23.263434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb42039-a378-4711-a34c-3f3f4a7a29c2 None admin] PATH_INFO: `/v3/groups/ee669194886e4a1c9d35bf7d4fb62d91` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:23.265190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb42039-a378-4711-a34c-3f3f4a7a29c2 None admin] RBAC: Authorizing `identity:get_group(group_id=ee669194886e4a1c9d35bf7d4fb62d91)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:23.267182 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb42039-a378-4711-a34c-3f3f4a7a29c2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:23.269752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 213/434] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 15:58:23 2022] GET /identity/v3/groups/ee669194886e4a1c9d35bf7d4fb62d91 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:23.271511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8be5a4-b04f-4e23-a328-928dd2ae867e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:23.272074 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8be5a4-b04f-4e23-a328-928dd2ae867e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:23.272663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8be5a4-b04f-4e23-a328-928dd2ae867e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:23.291828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-2c8be5a4-b04f-4e23-a328-928dd2ae867e None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:23.293731 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2c8be5a4-b04f-4e23-a328-928dd2ae867e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:23.320583 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb9e3ee-0b77-4037-9b8a-67999a503960 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:23.369374 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80348c7a-73c7-4fbb-a294-273ad3cd80cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:23.370494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80348c7a-73c7-4fbb-a294-273ad3cd80cc None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:23.370984 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80348c7a-73c7-4fbb-a294-273ad3cd80cc None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:23.371526 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80348c7a-73c7-4fbb-a294-273ad3cd80cc None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:23.372661 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80348c7a-73c7-4fbb-a294-273ad3cd80cc None admin] RBAC: Adding query filter params (name=member) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:23.373387 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80348c7a-73c7-4fbb-a294-273ad3cd80cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:23.375218 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80348c7a-73c7-4fbb-a294-273ad3cd80cc None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:23.382509 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:23.385817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 214/435] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:23.386312 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 222/436] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 15:58:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:23.386451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8be5a4-b04f-4e23-a328-928dd2ae867e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:23.394798 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80348c7a-73c7-4fbb-a294-273ad3cd80cc None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:23.494696 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:23.538512 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8be5a4-b04f-4e23-a328-928dd2ae867e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:23.548903 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80348c7a-73c7-4fbb-a294-273ad3cd80cc None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:23.601392 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4975f67-39f9-496a-b0dc-d6794170946b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:23.601392 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4975f67-39f9-496a-b0dc-d6794170946b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:23.601392 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4975f67-39f9-496a-b0dc-d6794170946b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:23.601392 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4975f67-39f9-496a-b0dc-d6794170946b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:23.602406 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4975f67-39f9-496a-b0dc-d6794170946b None admin] RBAC: Adding query filter params (group.id=ee669194886e4a1c9d35bf7d4fb62d91, role.id=e0b0c216ef51441ab34b6e036da09daa, scope.project.id=a83dc386fda8411eb2d4aef8e43b7065) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:23.602846 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4975f67-39f9-496a-b0dc-d6794170946b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:23.605389 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4975f67-39f9-496a-b0dc-d6794170946b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:23.610407 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1715c16d-4efb-4790-930a-3d4a2b81a1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:23.611709 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715c16d-4efb-4790-930a-3d4a2b81a1aa None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:23.611709 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715c16d-4efb-4790-930a-3d4a2b81a1aa None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:23.611709 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715c16d-4efb-4790-930a-3d4a2b81a1aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:23.617163 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 215/437] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jun 16 15:58:23 2022] GET /identity/v3/role_assignments?group.id=ee669194886e4a1c9d35bf7d4fb62d91&scope.project.id=a83dc386fda8411eb2d4aef8e43b7065&role.id=e0b0c216ef51441ab34b6e036da09daa => generated 611 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:23.618543 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4975f67-39f9-496a-b0dc-d6794170946b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:23.714181 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:23.719281 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1715c16d-4efb-4790-930a-3d4a2b81a1aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:23.722868 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1715c16d-4efb-4790-930a-3d4a2b81a1aa None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:23.817161 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4975f67-39f9-496a-b0dc-d6794170946b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:23.875168 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca7f98-dab0-4576-b69d-ba20cf3864ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:23.876249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca7f98-dab0-4576-b69d-ba20cf3864ae None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:23.876535 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca7f98-dab0-4576-b69d-ba20cf3864ae None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:23.876813 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca7f98-dab0-4576-b69d-ba20cf3864ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:23.880842 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 223/438] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:23.882068 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b25b31-33fa-4b65-8c3a-660b77be12c2 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:23.882361 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b25b31-33fa-4b65-8c3a-660b77be12c2 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:23.882891 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b25b31-33fa-4b65-8c3a-660b77be12c2 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:23.884318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 224/439] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:23.885117 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:23.889840 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca7f98-dab0-4576-b69d-ba20cf3864ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:23.890912 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21df90c-7888-4aa6-9f09-7525e375e25e None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:23.891197 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21df90c-7888-4aa6-9f09-7525e375e25e None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:23.891465 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21df90c-7888-4aa6-9f09-7525e375e25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:23.899166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-5dca7f98-dab0-4576-b69d-ba20cf3864ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:58:23.899166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:23.899166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:23.899166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:23.899166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:23.899166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:23.899166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:23.899166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:23.899166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:23.899166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:23.899166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:23.899166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:23.899166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:23.899166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:23.899166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:23.899166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:23.905504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:23.905504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:23.905504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:23.905504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:23.905504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:23.905504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:23.905504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:23.905504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:23.905504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:23.905504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:23.905504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:23.905504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:23.905504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:23.905504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:23.905504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:23.905504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:23.906451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:23.906451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:23.906451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:23.906451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:23.906451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:23.906451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:23.906451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:23.906451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:23.906451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:23.906451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:23.906451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:58:23.906451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:23.906451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 216/440] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 15:58:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:23.906451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca7f98-dab0-4576-b69d-ba20cf3864ae None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:23.915585 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-a21df90c-7888-4aa6-9f09-7525e375e25e None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:23.917687 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-a21df90c-7888-4aa6-9f09-7525e375e25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:24.009697 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 225/441] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:58:24.012891 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21df90c-7888-4aa6-9f09-7525e375e25e None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:24.047169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:24.056813 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca7f98-dab0-4576-b69d-ba20cf3864ae None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:24.098162 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a2a8e-c1a2-40a3-b396-fd12e706638b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:24.099034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a2a8e-c1a2-40a3-b396-fd12e706638b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:24.099347 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a2a8e-c1a2-40a3-b396-fd12e706638b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:24.099798 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a2a8e-c1a2-40a3-b396-fd12e706638b None admin] PATH_INFO: `/v3/projects/328b693aa62e4467aa34a749006ad00f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:24.101145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a2a8e-c1a2-40a3-b396-fd12e706638b None admin] RBAC: Authorizing `identity:get_project(project_id=328b693aa62e4467aa34a749006ad00f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:24.102664 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a2a8e-c1a2-40a3-b396-fd12e706638b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:24.109686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 217/442] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:23 2022] GET /identity/v3/projects/328b693aa62e4467aa34a749006ad00f => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:24.109686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a2a8e-c1a2-40a3-b396-fd12e706638b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:24.131451 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21df90c-7888-4aa6-9f09-7525e375e25e None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:24.180378 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:24.183524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f35de-6bba-4ded-bff7-953b908a07b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:24.184214 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f35de-6bba-4ded-bff7-953b908a07b0 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:24.184499 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f35de-6bba-4ded-bff7-953b908a07b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:24.184776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f35de-6bba-4ded-bff7-953b908a07b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:24.185596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f35de-6bba-4ded-bff7-953b908a07b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:24.185983 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f35de-6bba-4ded-bff7-953b908a07b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:24.187513 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f35de-6bba-4ded-bff7-953b908a07b0 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:24.196483 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 226/443] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:24.197326 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f35de-6bba-4ded-bff7-953b908a07b0 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:24.222044 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a2a8e-c1a2-40a3-b396-fd12e706638b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:24.259451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd157025-15c9-45af-8898-d01725825a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:24.260251 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd157025-15c9-45af-8898-d01725825a92 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:24.260530 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd157025-15c9-45af-8898-d01725825a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:24.260803 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd157025-15c9-45af-8898-d01725825a92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:24.272123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd157025-15c9-45af-8898-d01725825a92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:24.281091 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-dd157025-15c9-45af-8898-d01725825a92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:24.281091 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:24.281091 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:24.281091 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:24.281091 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:24.281091 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:24.281091 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:24.281091 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:24.281091 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:24.281091 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:24.281091 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:24.281091 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:24.281091 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:24.281091 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:24.281091 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:24.281091 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:24.282201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:24.282201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:24.282201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:24.282201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:24.282201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:24.282201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:24.282201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:24.282201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:24.282201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:24.282201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:24.282201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:24.282201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:24.282201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:24.282201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:24.282201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:24.282201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:24.283102 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:24.283102 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:24.283102 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:24.283102 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:24.283102 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:24.283102 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:24.283102 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:24.283102 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:24.283102 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:24.283102 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:24.283102 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:24.283102 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:24.286242 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 218/444] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 15:58:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:24.287024 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd157025-15c9-45af-8898-d01725825a92 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:24.304667 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:24.317501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f35de-6bba-4ded-bff7-953b908a07b0 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:24.355367 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946dc3b-47c2-4c7a-a659-496c39056b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:24.356223 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946dc3b-47c2-4c7a-a659-496c39056b48 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:24.356500 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946dc3b-47c2-4c7a-a659-496c39056b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:24.356768 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946dc3b-47c2-4c7a-a659-496c39056b48 None admin] PATH_INFO: `/v3/groups/ee669194886e4a1c9d35bf7d4fb62d91` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:24.359074 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e946dc3b-47c2-4c7a-a659-496c39056b48 None admin] RBAC: Authorizing `identity:get_group(group_id=ee669194886e4a1c9d35bf7d4fb62d91)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:24.360542 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e946dc3b-47c2-4c7a-a659-496c39056b48 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:24.362240 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 227/445] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 15:58:24 2022] GET /identity/v3/groups/ee669194886e4a1c9d35bf7d4fb62d91 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:24.363022 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946dc3b-47c2-4c7a-a659-496c39056b48 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:24.409092 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd157025-15c9-45af-8898-d01725825a92 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:24.439576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:24.451522 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7320d-7bf6-4b81-b793-02665bee8a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:24.452257 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7320d-7bf6-4b81-b793-02665bee8a83 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:24.452705 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7320d-7bf6-4b81-b793-02665bee8a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:24.453049 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7320d-7bf6-4b81-b793-02665bee8a83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:24.489591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946dc3b-47c2-4c7a-a659-496c39056b48 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:24.525508 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82429825-efa6-4917-b924-bd292c8fc08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:24.526214 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82429825-efa6-4917-b924-bd292c8fc08b None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:24.526502 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82429825-efa6-4917-b924-bd292c8fc08b None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:24.526788 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82429825-efa6-4917-b924-bd292c8fc08b None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:24.527652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82429825-efa6-4917-b924-bd292c8fc08b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:24.528003 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82429825-efa6-4917-b924-bd292c8fc08b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:24.530749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82429825-efa6-4917-b924-bd292c8fc08b None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:24.531704 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7320d-7bf6-4b81-b793-02665bee8a83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:24.532822 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7320d-7bf6-4b81-b793-02665bee8a83 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:24.539458 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 228/446] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:24.540508 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82429825-efa6-4917-b924-bd292c8fc08b None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:24.563399 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:24.635706 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 219/447] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:24.636639 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7320d-7bf6-4b81-b793-02665bee8a83 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:24.645909 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82429825-efa6-4917-b924-bd292c8fc08b None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:24.666107 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:24.681175 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b433b9-be39-49ab-9501-ae88f421d47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:24.681844 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b433b9-be39-49ab-9501-ae88f421d47e None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:24.682163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b433b9-be39-49ab-9501-ae88f421d47e None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:24.682452 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b433b9-be39-49ab-9501-ae88f421d47e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:24.683280 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b433b9-be39-49ab-9501-ae88f421d47e None admin] RBAC: Adding query filter params (group.id=ee669194886e4a1c9d35bf7d4fb62d91, role.id=3ce570490865493380e6941d004136bd, scope.project.id=328b693aa62e4467aa34a749006ad00f) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:24.683595 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b433b9-be39-49ab-9501-ae88f421d47e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:24.685066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b433b9-be39-49ab-9501-ae88f421d47e None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:24.700532 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 229/448] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jun 16 15:58:24 2022] GET /identity/v3/role_assignments?group.id=ee669194886e4a1c9d35bf7d4fb62d91&scope.project.id=328b693aa62e4467aa34a749006ad00f&role.id=3ce570490865493380e6941d004136bd => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:24.700532 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b433b9-be39-49ab-9501-ae88f421d47e None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:24.756108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7320d-7bf6-4b81-b793-02665bee8a83 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:24.812329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18615cde-c14f-4845-b6f3-8171e27ae0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:24.813008 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18615cde-c14f-4845-b6f3-8171e27ae0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:24.813300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18615cde-c14f-4845-b6f3-8171e27ae0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:24.813578 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18615cde-c14f-4845-b6f3-8171e27ae0d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:24.823698 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:24.845727 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b433b9-be39-49ab-9501-ae88f421d47e None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:24.898635 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea7f1b-bf5d-48d9-8bed-692cd3b1c1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:24.900721 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea7f1b-bf5d-48d9-8bed-692cd3b1c1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:24.900721 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea7f1b-bf5d-48d9-8bed-692cd3b1c1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:24.900721 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea7f1b-bf5d-48d9-8bed-692cd3b1c1a5 None admin] PATH_INFO: `/v3/projects/328b693aa62e4467aa34a749006ad00f` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:24.901398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cea7f1b-bf5d-48d9-8bed-692cd3b1c1a5 None admin] RBAC: Authorizing `identity:get_project(project_id=328b693aa62e4467aa34a749006ad00f)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:24.902934 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cea7f1b-bf5d-48d9-8bed-692cd3b1c1a5 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:24.904757 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 230/449] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:24 2022] GET /identity/v3/projects/328b693aa62e4467aa34a749006ad00f => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:24.910425 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea7f1b-bf5d-48d9-8bed-692cd3b1c1a5 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:24.920846 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18615cde-c14f-4845-b6f3-8171e27ae0d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:24.922765 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18615cde-c14f-4845-b6f3-8171e27ae0d1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:24.969776 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:25.064295 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea7f1b-bf5d-48d9-8bed-692cd3b1c1a5 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:25.082246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 220/450] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:25.088915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18615cde-c14f-4845-b6f3-8171e27ae0d1 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:25.111781 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb8aa1-2732-4f60-8de0-fcfb1950d597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:25.113218 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb8aa1-2732-4f60-8de0-fcfb1950d597 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:25.113675 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb8aa1-2732-4f60-8de0-fcfb1950d597 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:25.114172 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb8aa1-2732-4f60-8de0-fcfb1950d597 None admin] PATH_INFO: `/v3/groups/ee669194886e4a1c9d35bf7d4fb62d91` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:25.115906 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb8aa1-2732-4f60-8de0-fcfb1950d597 None admin] RBAC: Authorizing `identity:get_group(group_id=ee669194886e4a1c9d35bf7d4fb62d91)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:25.117571 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb8aa1-2732-4f60-8de0-fcfb1950d597 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:25.120409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 231/451] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 15:58:24 2022] GET /identity/v3/groups/ee669194886e4a1c9d35bf7d4fb62d91 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:25.127440 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb8aa1-2732-4f60-8de0-fcfb1950d597 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:25.156576 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:25.210597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18615cde-c14f-4845-b6f3-8171e27ae0d1 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:25.237116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:25.243035 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb8aa1-2732-4f60-8de0-fcfb1950d597 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:25.246661 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97304772-bdcd-4346-bae1-0594e730be73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:25.247521 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97304772-bdcd-4346-bae1-0594e730be73 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:25.247907 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97304772-bdcd-4346-bae1-0594e730be73 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:25.249069 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97304772-bdcd-4346-bae1-0594e730be73 None admin] PATH_INFO: `/v3/projects/47ead53fd5cc424c8de9383e832d40a2` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:25.250258 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97304772-bdcd-4346-bae1-0594e730be73 None admin] RBAC: Authorizing `identity:get_project(project_id=47ead53fd5cc424c8de9383e832d40a2)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:25.251725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97304772-bdcd-4346-bae1-0594e730be73 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:25.253432 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 221/452] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:25 2022] GET /identity/v3/projects/47ead53fd5cc424c8de9383e832d40a2 => generated 344 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:25.259064 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97304772-bdcd-4346-bae1-0594e730be73 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:25.277858 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0ffc6-dc5a-4f74-8050-28ed2f5f90e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:25.278613 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0ffc6-dc5a-4f74-8050-28ed2f5f90e8 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:25.278907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0ffc6-dc5a-4f74-8050-28ed2f5f90e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:25.279226 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0ffc6-dc5a-4f74-8050-28ed2f5f90e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:25.280741 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe0ffc6-dc5a-4f74-8050-28ed2f5f90e8 None admin] RBAC: Adding query filter params (group.id=ee669194886e4a1c9d35bf7d4fb62d91, role.id=e0b0c216ef51441ab34b6e036da09daa, scope.project.id=328b693aa62e4467aa34a749006ad00f) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:25.281286 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe0ffc6-dc5a-4f74-8050-28ed2f5f90e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:25.282970 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe0ffc6-dc5a-4f74-8050-28ed2f5f90e8 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:25.288870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 232/453] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jun 16 15:58:25 2022] GET /identity/v3/role_assignments?group.id=ee669194886e4a1c9d35bf7d4fb62d91&scope.project.id=328b693aa62e4467aa34a749006ad00f&role.id=e0b0c216ef51441ab34b6e036da09daa => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:25.366172 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97304772-bdcd-4346-bae1-0594e730be73 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:25.402120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c65f36-0176-4f74-9e6e-9bff220d970e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:25.402994 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c65f36-0176-4f74-9e6e-9bff220d970e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:25.403283 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c65f36-0176-4f74-9e6e-9bff220d970e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:25.403564 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c65f36-0176-4f74-9e6e-9bff220d970e None admin] PATH_INFO: `/v3/groups/15b9a6ed0fdb440e93c60902d1603904` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:25.404866 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c65f36-0176-4f74-9e6e-9bff220d970e None admin] RBAC: Authorizing `identity:get_group(group_id=15b9a6ed0fdb440e93c60902d1603904)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:25.406382 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c65f36-0176-4f74-9e6e-9bff220d970e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:25.408071 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 222/454] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 15:58:25 2022] GET /identity/v3/groups/15b9a6ed0fdb440e93c60902d1603904 => generated 229 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:25.414208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0ffc6-dc5a-4f74-8050-28ed2f5f90e8 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:25.503290 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:25.520138 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0ffc6-dc5a-4f74-8050-28ed2f5f90e8 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:25.554077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe7bbf-f894-4657-8808-f8970df51c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:25.554741 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe7bbf-f894-4657-8808-f8970df51c0b None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:25.555029 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe7bbf-f894-4657-8808-f8970df51c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:25.555306 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe7bbf-f894-4657-8808-f8970df51c0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:25.556143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fe7bbf-f894-4657-8808-f8970df51c0b None admin] RBAC: Adding query filter params (group.id=15b9a6ed0fdb440e93c60902d1603904, role.id=bbbc77ec645d4ff5a8bf29ce0377f578, scope.project.id=47ead53fd5cc424c8de9383e832d40a2) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:25.556459 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fe7bbf-f894-4657-8808-f8970df51c0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:25.557910 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fe7bbf-f894-4657-8808-f8970df51c0b None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:25.564662 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 233/455] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jun 16 15:58:25 2022] GET /identity/v3/role_assignments?group.id=15b9a6ed0fdb440e93c60902d1603904&scope.project.id=47ead53fd5cc424c8de9383e832d40a2&role.id=bbbc77ec645d4ff5a8bf29ce0377f578 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:28.072245 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e89a3-3f92-4d7c-a5b7-f9819dad075b None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:28.072786 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e89a3-3f92-4d7c-a5b7-f9819dad075b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:28.072786 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e89a3-3f92-4d7c-a5b7-f9819dad075b None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:28.074205 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 223/456] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:28.080145 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abe6d3-b1a2-485d-9d1b-cdf9f1a55737 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:28.080612 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abe6d3-b1a2-485d-9d1b-cdf9f1a55737 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:28.080695 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abe6d3-b1a2-485d-9d1b-cdf9f1a55737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:28.108013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-56abe6d3-b1a2-485d-9d1b-cdf9f1a55737 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:28.109324 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-56abe6d3-b1a2-485d-9d1b-cdf9f1a55737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:28.208634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 234/457] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:28.255498 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e89a3-3f92-4d7c-a5b7-f9819dad075b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:28.292954 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa95fa-5114-43d5-8384-81e8dbe4103b None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:28.293759 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa95fa-5114-43d5-8384-81e8dbe4103b None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:28.294619 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa95fa-5114-43d5-8384-81e8dbe4103b None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:28.296983 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 235/458] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:58:28.302276 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9574a52-c750-428d-a782-6a0d3c20298e None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:28.302808 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9574a52-c750-428d-a782-6a0d3c20298e None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:28.303306 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9574a52-c750-428d-a782-6a0d3c20298e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:28.334381 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-d9574a52-c750-428d-a782-6a0d3c20298e None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:28.335806 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-d9574a52-c750-428d-a782-6a0d3c20298e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:28.344902 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:28.415904 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e89a3-3f92-4d7c-a5b7-f9819dad075b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:28.464691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 236/459] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:58:28.488480 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a52463-b0df-4c62-bae0-1326c02517d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:28.489311 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a52463-b0df-4c62-bae0-1326c02517d8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:28.489598 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a52463-b0df-4c62-bae0-1326c02517d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:28.490071 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a52463-b0df-4c62-bae0-1326c02517d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:28.491188 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9574a52-c750-428d-a782-6a0d3c20298e None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:28.504791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:28.522181 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a52463-b0df-4c62-bae0-1326c02517d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:28.523689 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a52463-b0df-4c62-bae0-1326c02517d8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:28.564850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-28a52463-b0df-4c62-bae0-1326c02517d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:28.564850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:28.564850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:28.564850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:28.564850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:28.564850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:28.564850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:28.564850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:28.564850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:28.564850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:28.564850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:28.564850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:28.564850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:28.564850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:28.564850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:28.565811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:28.565811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:28.565811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:28.565811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:28.565811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:28.565811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:28.565811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:28.565811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:28.565811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:28.565811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:28.565811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:28.565811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:28.565811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:28.565811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:28.565811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:28.565811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:28.565811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:28.566699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:28.566699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:28.566699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:28.566699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:28.568409 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 224/460] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:28.577129 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a52463-b0df-4c62-bae0-1326c02517d8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:28.638866 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9574a52-c750-428d-a782-6a0d3c20298e None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:28.680779 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cffdbf2-5c62-4d7e-96fe-7d84fd35f5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:28.682001 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffdbf2-5c62-4d7e-96fe-7d84fd35f5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:28.682491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffdbf2-5c62-4d7e-96fe-7d84fd35f5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:28.682967 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffdbf2-5c62-4d7e-96fe-7d84fd35f5d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:28.694839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cffdbf2-5c62-4d7e-96fe-7d84fd35f5d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:28.696654 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cffdbf2-5c62-4d7e-96fe-7d84fd35f5d1 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:28.705107 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a52463-b0df-4c62-bae0-1326c02517d8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:28.718006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-8cffdbf2-5c62-4d7e-96fe-7d84fd35f5d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:28.718006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:28.718006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:28.718006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:28.718006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:28.718006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:28.718006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:28.718006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:28.718006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:28.718006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:28.718006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:28.718006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:28.718006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:28.718006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:28.718006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:28.719066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:28.719066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:28.719066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:28.719066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:28.719066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:28.719066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:28.719066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:28.719066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:28.719066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:28.719066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:28.719066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:28.719066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:28.719066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:28.719066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:28.719066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:28.719066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:28.719066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:28.720018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:28.720018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:28.720018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:28.720018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:28.722010 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:28.723697 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 237/461] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:58:28.725473 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7087f1-506a-4872-b810-866aa13d822a None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:28.726177 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7087f1-506a-4872-b810-866aa13d822a None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:28.727388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7087f1-506a-4872-b810-866aa13d822a None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:28.729032 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 238/462] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:28.730058 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7087f1-506a-4872-b810-866aa13d822a None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:28.761879 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3bb96-bc93-4a2c-9b3a-8377676f0599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:28.762742 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3bb96-bc93-4a2c-9b3a-8377676f0599 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:28.763029 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3bb96-bc93-4a2c-9b3a-8377676f0599 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:28.763771 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3bb96-bc93-4a2c-9b3a-8377676f0599 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:28.765470 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c3bb96-bc93-4a2c-9b3a-8377676f0599 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:28.765780 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c3bb96-bc93-4a2c-9b3a-8377676f0599 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:28.767860 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c3bb96-bc93-4a2c-9b3a-8377676f0599 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:28.801437 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 225/463] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:28.803201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfb086-7df9-4965-9ec0-a1801f4113ab None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:28.803725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfb086-7df9-4965-9ec0-a1801f4113ab None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:28.804203 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfb086-7df9-4965-9ec0-a1801f4113ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:28.819165 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:28.827633 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-5bbfb086-7df9-4965-9ec0-a1801f4113ab None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:28.829975 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5bbfb086-7df9-4965-9ec0-a1801f4113ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:28.872207 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7087f1-506a-4872-b810-866aa13d822a None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:28.919595 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 226/464] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:28.921281 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfb086-7df9-4965-9ec0-a1801f4113ab None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:28.927389 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a81363-ee3b-42b6-a67d-86dcac9aebe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:28.928604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a81363-ee3b-42b6-a67d-86dcac9aebe8 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:28.930794 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a81363-ee3b-42b6-a67d-86dcac9aebe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:28.931277 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a81363-ee3b-42b6-a67d-86dcac9aebe8 None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:28.933843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a81363-ee3b-42b6-a67d-86dcac9aebe8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:28.934385 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a81363-ee3b-42b6-a67d-86dcac9aebe8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:28.937047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a81363-ee3b-42b6-a67d-86dcac9aebe8 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:28.957172 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 239/465] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:28.958353 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a81363-ee3b-42b6-a67d-86dcac9aebe8 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:29.080949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfb086-7df9-4965-9ec0-a1801f4113ab None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:29.082153 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a81363-ee3b-42b6-a67d-86dcac9aebe8 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:29.083677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:29.113293 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:29.137482 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa090924-5753-4813-ab95-0373c8e1491f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:29.137948 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5766353e-b583-4aaa-9732-5178299cdfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:29.138206 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa090924-5753-4813-ab95-0373c8e1491f None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:29.138495 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa090924-5753-4813-ab95-0373c8e1491f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:29.138807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa090924-5753-4813-ab95-0373c8e1491f None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:29.139535 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa090924-5753-4813-ab95-0373c8e1491f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:29.139936 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766353e-b583-4aaa-9732-5178299cdfd4 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:29.140439 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766353e-b583-4aaa-9732-5178299cdfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:29.140904 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766353e-b583-4aaa-9732-5178299cdfd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:29.141004 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa090924-5753-4813-ab95-0373c8e1491f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:29.154301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5766353e-b583-4aaa-9732-5178299cdfd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:29.156312 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5766353e-b583-4aaa-9732-5178299cdfd4 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:29.168513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 227/466] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 15:58:28 2022] POST /identity/v3/users => generated 246 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:58:29.169744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa090924-5753-4813-ab95-0373c8e1491f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:29.181881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-5766353e-b583-4aaa-9732-5178299cdfd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:29.181881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:29.181881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:29.181881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:29.181881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:29.181881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:29.181881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:29.181881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:29.181881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:29.181881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:29.181881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:29.181881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:29.181881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:29.181881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:29.181881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:29.182814 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:29.182814 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:29.182814 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:29.182814 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:29.182814 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:29.182814 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:29.182814 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:29.182814 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:29.182814 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:29.182814 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:29.182814 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:29.182814 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:29.182814 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:29.182814 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:29.182814 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:29.182814 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:29.182814 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:29.183639 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:29.183639 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:29.183639 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:29.183639 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:29.187371 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 240/467] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:29.189232 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770ddf2-1d62-4eb5-973c-c8d41590d7c7 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:29.189744 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770ddf2-1d62-4eb5-973c-c8d41590d7c7 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:29.190269 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770ddf2-1d62-4eb5-973c-c8d41590d7c7 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:29.193022 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 241/468] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:29.194522 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770ddf2-1d62-4eb5-973c-c8d41590d7c7 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:29.340978 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa090924-5753-4813-ab95-0373c8e1491f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:29.342211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770ddf2-1d62-4eb5-973c-c8d41590d7c7 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:29.390514 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ccf87-481d-4eb9-8af2-5eafa02d8dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:29.391185 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ccf87-481d-4eb9-8af2-5eafa02d8dab None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:29.391475 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ccf87-481d-4eb9-8af2-5eafa02d8dab None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:29.391758 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ccf87-481d-4eb9-8af2-5eafa02d8dab None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:29.392500 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ccf87-481d-4eb9-8af2-5eafa02d8dab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:29.392812 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ccf87-481d-4eb9-8af2-5eafa02d8dab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:29.395038 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ccf87-481d-4eb9-8af2-5eafa02d8dab None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:29.395166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351f950-9994-47e9-a880-dbb5677587b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:29.395423 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351f950-9994-47e9-a880-dbb5677587b7 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:29.395636 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351f950-9994-47e9-a880-dbb5677587b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:29.395810 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351f950-9994-47e9-a880-dbb5677587b7 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:29.396553 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351f950-9994-47e9-a880-dbb5677587b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:29.399008 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351f950-9994-47e9-a880-dbb5677587b7 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:29.410357 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 242/469] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:29.410357 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed0461-0ba8-43be-a166-9226163ada68 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:29.410357 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed0461-0ba8-43be-a166-9226163ada68 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:29.410357 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed0461-0ba8-43be-a166-9226163ada68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:29.425722 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 228/470] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 15:58:29 2022] POST /identity/v3/users => generated 248 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:58:29.427393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351f950-9994-47e9-a880-dbb5677587b7 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:29.439168 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-38ed0461-0ba8-43be-a166-9226163ada68 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:29.440434 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-38ed0461-0ba8-43be-a166-9226163ada68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:29.456019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:29.480131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:29.555140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 243/471] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:29.598717 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed0461-0ba8-43be-a166-9226163ada68 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:29.604061 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351f950-9994-47e9-a880-dbb5677587b7 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:29.673463 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1534e6b-c446-4dbd-a09c-b5a0fc5df185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:29.674480 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1534e6b-c446-4dbd-a09c-b5a0fc5df185 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:29.674770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1534e6b-c446-4dbd-a09c-b5a0fc5df185 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:29.675571 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1534e6b-c446-4dbd-a09c-b5a0fc5df185 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:29.676632 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1534e6b-c446-4dbd-a09c-b5a0fc5df185 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:29.678758 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1534e6b-c446-4dbd-a09c-b5a0fc5df185 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:29.710166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 229/472] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 15:58:29 2022] POST /identity/v3/users => generated 248 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:58:29.781859 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed0461-0ba8-43be-a166-9226163ada68 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:29.822354 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d31a46-6af6-4ccb-80e4-949f9dafc752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:29.823232 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d31a46-6af6-4ccb-80e4-949f9dafc752 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:29.823524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d31a46-6af6-4ccb-80e4-949f9dafc752 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:29.823804 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d31a46-6af6-4ccb-80e4-949f9dafc752 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:29.836720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d31a46-6af6-4ccb-80e4-949f9dafc752 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:29.838307 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d31a46-6af6-4ccb-80e4-949f9dafc752 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:29.860624 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-e3d31a46-6af6-4ccb-80e4-949f9dafc752 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:29.860624 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:29.860624 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:29.860624 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:29.860624 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:29.860624 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:29.860624 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:29.860624 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:29.860624 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:29.860624 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:29.860624 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:29.860624 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:29.860624 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:29.860624 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:29.860624 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:29.861544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:29.861544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:29.861544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:29.861544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:29.861544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:29.861544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:29.861544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:29.861544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:29.861544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:29.861544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:29.861544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:29.861544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:29.861544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:29.861544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:29.861544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:29.861544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:29.861544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:29.862442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:29.862442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:29.862442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:29.862442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:29.862442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 244/473] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:29.869240 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1534e6b-c446-4dbd-a09c-b5a0fc5df185 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:29.935305 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ca1b4-f40e-47de-a30d-d6eebddbbf54 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:29.935662 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ca1b4-f40e-47de-a30d-d6eebddbbf54 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:29.936037 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ca1b4-f40e-47de-a30d-d6eebddbbf54 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:29.938692 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 245/474] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:29.944753 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f8ee9-e2d4-4928-b1da-7228d952fd32 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:29.945044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f8ee9-e2d4-4928-b1da-7228d952fd32 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:29.945916 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f8ee9-e2d4-4928-b1da-7228d952fd32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:29.955821 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:29.969969 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-8e6f8ee9-e2d4-4928-b1da-7228d952fd32 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:29.971164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-8e6f8ee9-e2d4-4928-b1da-7228d952fd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:29.981610 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:30.000206 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1534e6b-c446-4dbd-a09c-b5a0fc5df185 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:30.040712 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c94e5-fc56-4141-8931-e6afa0008657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:30.041387 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c94e5-fc56-4141-8931-e6afa0008657 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:30.041670 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c94e5-fc56-4141-8931-e6afa0008657 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:30.041954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c94e5-fc56-4141-8931-e6afa0008657 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:30.042684 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c94e5-fc56-4141-8931-e6afa0008657 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:30.043086 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c94e5-fc56-4141-8931-e6afa0008657 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:30.044613 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c94e5-fc56-4141-8931-e6afa0008657 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:30.048568 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 246/475] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:58:30.060096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 230/476] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:30.066291 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f8ee9-e2d4-4928-b1da-7228d952fd32 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:30.079610 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c94e5-fc56-4141-8931-e6afa0008657 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:30.223287 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c94e5-fc56-4141-8931-e6afa0008657 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:30.257366 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f8ee9-e2d4-4928-b1da-7228d952fd32 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:30.259541 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:30.272537 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b5676-1846-400d-92d8-607005918c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:30.273555 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b5676-1846-400d-92d8-607005918c2d None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:30.274066 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b5676-1846-400d-92d8-607005918c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:30.274543 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b5676-1846-400d-92d8-607005918c2d None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:30.275517 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b5676-1846-400d-92d8-607005918c2d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:30.277245 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b5676-1846-400d-92d8-607005918c2d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:30.289102 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.sql.core [None req-5f4b5676-1846-400d-92d8-607005918c2d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 15:58:30.289102 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 15:58:30.289102 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [parameters: {'id': 'b7855bf9a56f4d529938d891bcb44869', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 15:58:30.289102 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80440) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 15:58:30.305803 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b5bbd-f3e6-478a-ad35-83b2f9072ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:30.306529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b5bbd-f3e6-478a-ad35-83b2f9072ed8 None admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:30.306822 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b5bbd-f3e6-478a-ad35-83b2f9072ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:30.307102 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b5bbd-f3e6-478a-ad35-83b2f9072ed8 None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:30.307968 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b5bbd-f3e6-478a-ad35-83b2f9072ed8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:30.309485 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b5bbd-f3e6-478a-ad35-83b2f9072ed8 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:30.321966 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-5f4b5676-1846-400d-92d8-607005918c2d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 15:58:30.321966 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:30.321966 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 15:58:30.321966 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 15:58:30.321966 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 15:58:30.321966 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 15:58:30.321966 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 15:58:30.321966 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 15:58:30.321966 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 15:58:30.321966 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application conn.query(q) Jun 16 15:58:30.321966 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 15:58:30.321966 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 15:58:30.321966 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 15:58:30.321966 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application result.read() Jun 16 15:58:30.321966 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 15:58:30.323070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 15:58:30.323070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 15:58:30.323070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 15:58:30.323070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 15:58:30.323070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 15:58:30.323070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 15:58:30.323070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 15:58:30.323070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 15:58:30.323070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:30.323070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 15:58:30.323070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:30.323070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:30.323070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 15:58:30.323070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 15:58:30.323070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 16 15:58:30.323070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return ref.to_dict() Jun 16 15:58:30.323070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 15:58:30.323070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application next(self.gen) Jun 16 15:58:30.324264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 15:58:30.324264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application yield resource Jun 16 15:58:30.324264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 15:58:30.324264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application next(self.gen) Jun 16 15:58:30.324264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 15:58:30.324264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 15:58:30.324264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 15:58:30.324264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 15:58:30.324264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 15:58:30.324264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise self.value Jun 16 15:58:30.324264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 15:58:30.324264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 15:58:30.324264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 15:58:30.324264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application session.commit() Jun 16 15:58:30.324264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 15:58:30.324264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 15:58:30.324264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 15:58:30.325323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 15:58:30.325323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 15:58:30.325323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self.session.flush() Jun 16 15:58:30.325323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 15:58:30.325323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 15:58:30.325323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 15:58:30.325323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 15:58:30.325323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 15:58:30.325323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application compat.raise_( Jun 16 15:58:30.325323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 15:58:30.325323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception Jun 16 15:58:30.325323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 15:58:30.325323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 15:58:30.325323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 15:58:30.325323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 15:58:30.325323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 15:58:30.325323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 15:58:30.327019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 15:58:30.327019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 15:58:30.327019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 15:58:30.327019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 15:58:30.327019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 15:58:30.327019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 15:58:30.327019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 15:58:30.327019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 15:58:30.327019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 15:58:30.327019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 15:58:30.327019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 15:58:30.327019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 15:58:30.327019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 15:58:30.327019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 15:58:30.327019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 15:58:30.327019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception Jun 16 15:58:30.328014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 15:58:30.328014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 15:58:30.328014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 15:58:30.328014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 15:58:30.328014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 15:58:30.328014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 15:58:30.328014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 15:58:30.328014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application conn.query(q) Jun 16 15:58:30.328014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 15:58:30.328014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 15:58:30.328014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 15:58:30.328014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application result.read() Jun 16 15:58:30.328014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 15:58:30.328014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 15:58:30.328014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 15:58:30.328014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 15:58:30.328979 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 15:58:30.328979 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 15:58:30.328979 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 15:58:30.328979 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 15:58:30.328979 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 15:58:30.328979 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 15:58:30.328979 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [parameters: {'id': 'b7855bf9a56f4d529938d891bcb44869', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 15:58:30.328979 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 15:58:30.328979 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:30.328979 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:58:30.328979 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:30.328979 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:30.328979 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:30.328979 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:30.328979 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:30.328979 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:30.334379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:30.334379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:30.334379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:30.334379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:30.334379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:30.334379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:30.334379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 16 15:58:30.334379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 16 15:58:30.334379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:30.334379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:30.334379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 16 15:58:30.334379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 16 15:58:30.334379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 15:58:30.334379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 15:58:30.334379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 15:58:30.334379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:30.335346 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 231/477] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jun 16 15:58:30 2022] POST /identity/v3/roles => generated 144 bytes in 240 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:30.343277 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b5676-1846-400d-92d8-607005918c2d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:30.351234 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:30.351581 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 247/478] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 15:58:30 2022] POST /identity/v3/users => generated 249 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:58:30.504946 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b5676-1846-400d-92d8-607005918c2d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:30.543486 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e60e9-9122-4f29-830a-299b15aa6df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:30.544318 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e60e9-9122-4f29-830a-299b15aa6df0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:30.544603 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e60e9-9122-4f29-830a-299b15aa6df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:30.545489 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e60e9-9122-4f29-830a-299b15aa6df0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:30.557551 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e60e9-9122-4f29-830a-299b15aa6df0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:30.566740 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-136e60e9-9122-4f29-830a-299b15aa6df0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:58:30.566740 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:30.566740 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:30.566740 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:30.566740 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:30.566740 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:30.566740 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:30.566740 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:30.566740 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:30.566740 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:30.566740 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:30.566740 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:30.566740 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:30.566740 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:30.566740 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:30.566740 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:30.567719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:30.567719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:30.567719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:30.567719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:30.567719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:30.567719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:30.567719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:30.567719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:30.567719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:30.567719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:30.567719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:30.567719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:30.567719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:30.567719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:30.567719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:30.567719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:30.568518 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:30.568518 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:30.568518 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:30.568518 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:30.568518 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:30.568518 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:30.568518 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:30.568518 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:30.568518 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:30.568518 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:30.568518 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:58:30.568518 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:30.568518 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 232/479] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 15:58:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:30.574195 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b5bbd-f3e6-478a-ad35-83b2f9072ed8 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:30.717994 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b5bbd-f3e6-478a-ad35-83b2f9072ed8 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:30.753514 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e88f3-8a9d-480c-86fa-e2b396ee7117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:30.754249 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e88f3-8a9d-480c-86fa-e2b396ee7117 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:30.754540 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e88f3-8a9d-480c-86fa-e2b396ee7117 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:30.754822 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e88f3-8a9d-480c-86fa-e2b396ee7117 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:30.755638 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0e88f3-8a9d-480c-86fa-e2b396ee7117 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:30.755896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0e88f3-8a9d-480c-86fa-e2b396ee7117 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:30.757408 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0e88f3-8a9d-480c-86fa-e2b396ee7117 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:30.768695 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 248/480] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:31.789429 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4a40f-df88-45d7-bf35-9b2538935087 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:31.792871 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4a40f-df88-45d7-bf35-9b2538935087 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:31.793333 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4a40f-df88-45d7-bf35-9b2538935087 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:31.795902 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 233/481] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:31.801217 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904715c2-418c-4d85-a40e-7bc3ab8bd634 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:31.801689 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904715c2-418c-4d85-a40e-7bc3ab8bd634 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:31.801776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904715c2-418c-4d85-a40e-7bc3ab8bd634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:31.823800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-904715c2-418c-4d85-a40e-7bc3ab8bd634 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:31.825011 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-904715c2-418c-4d85-a40e-7bc3ab8bd634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:31.890221 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:31.910042 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 249/482] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:31.941473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4a40f-df88-45d7-bf35-9b2538935087 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:32.024921 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:32.099684 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4a40f-df88-45d7-bf35-9b2538935087 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:32.145693 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f283a07-4991-4992-b095-9f98dcd89fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:32.146563 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f283a07-4991-4992-b095-9f98dcd89fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:32.146848 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f283a07-4991-4992-b095-9f98dcd89fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:32.147125 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f283a07-4991-4992-b095-9f98dcd89fa0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:32.167861 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f283a07-4991-4992-b095-9f98dcd89fa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:32.176637 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-6f283a07-4991-4992-b095-9f98dcd89fa0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:32.176637 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:32.176637 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:32.176637 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:32.176637 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:32.176637 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:32.176637 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:32.176637 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:32.176637 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:32.176637 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:32.176637 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:32.176637 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:32.176637 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:32.176637 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:32.176637 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:32.176637 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:32.177969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:32.177969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:32.177969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:32.177969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:32.177969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:32.177969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:32.177969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:32.177969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:32.177969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:32.177969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:32.177969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:32.177969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:32.177969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:32.177969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:32.177969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:32.177969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:32.178851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:32.178851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:32.178851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:32.178851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:32.178851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:32.178851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:32.178851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:32.178851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:32.178851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:32.178851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:32.178851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:32.178851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:32.182363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 234/483] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 15:58:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:32.183659 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904715c2-418c-4d85-a40e-7bc3ab8bd634 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:32.314071 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904715c2-418c-4d85-a40e-7bc3ab8bd634 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:32.351980 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceeb93f-487f-45f1-8324-b50d85be7037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:32.352660 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceeb93f-487f-45f1-8324-b50d85be7037 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:32.352947 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceeb93f-487f-45f1-8324-b50d85be7037 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:32.353220 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceeb93f-487f-45f1-8324-b50d85be7037 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:32.354006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceeb93f-487f-45f1-8324-b50d85be7037 None admin] RBAC: Adding query filter params (name=service) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:32.354327 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceeb93f-487f-45f1-8324-b50d85be7037 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:32.355741 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceeb93f-487f-45f1-8324-b50d85be7037 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:32.369406 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 250/484] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 15:58:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:32.375024 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f283a07-4991-4992-b095-9f98dcd89fa0 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:32.386623 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090514b8-cf1d-4036-ab2f-a95b76af9dd1 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:32.386945 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090514b8-cf1d-4036-ab2f-a95b76af9dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:32.387184 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090514b8-cf1d-4036-ab2f-a95b76af9dd1 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:32.388917 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 251/485] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:58:32.394749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f978aa4-f173-4df3-a975-f47c32651344 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:32.395056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f978aa4-f173-4df3-a975-f47c32651344 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:32.395322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f978aa4-f173-4df3-a975-f47c32651344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:32.413769 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-0f978aa4-f173-4df3-a975-f47c32651344 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:32.415028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-0f978aa4-f173-4df3-a975-f47c32651344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:32.502286 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:32.547421 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 252/486] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:58:32.564262 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f283a07-4991-4992-b095-9f98dcd89fa0 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:32.582484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f978aa4-f173-4df3-a975-f47c32651344 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:32.594332 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:32.623001 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db80db-7fc5-4215-9422-e0683789a99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:32.623676 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db80db-7fc5-4215-9422-e0683789a99f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:32.623960 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db80db-7fc5-4215-9422-e0683789a99f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:32.624239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db80db-7fc5-4215-9422-e0683789a99f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:32.635802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db80db-7fc5-4215-9422-e0683789a99f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:32.637266 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db80db-7fc5-4215-9422-e0683789a99f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:32.655711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-80db80db-7fc5-4215-9422-e0683789a99f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:32.655711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:32.655711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:32.655711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:32.655711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:32.655711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:32.655711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:32.655711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:32.655711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:32.655711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:32.655711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:32.655711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:32.655711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:32.655711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:32.655711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:32.656615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:32.656615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:32.656615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:32.656615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:32.656615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:32.656615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:32.656615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:32.656615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:32.656615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:32.656615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:32.656615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:32.656615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:32.656615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:32.656615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:32.656615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:32.656615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:32.656615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:32.657454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:32.657454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:32.657454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:32.657454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:32.657454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 235/487] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:32.658178 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44dc1a7-2294-4541-be4d-ec6ff82e2726 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:32.658448 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44dc1a7-2294-4541-be4d-ec6ff82e2726 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:32.658734 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44dc1a7-2294-4541-be4d-ec6ff82e2726 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:32.660247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 236/488] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:32.663261 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44dc1a7-2294-4541-be4d-ec6ff82e2726 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:32.730033 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f978aa4-f173-4df3-a975-f47c32651344 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:32.769940 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb249c-b5e4-46a0-8d28-690bc2fe3b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:32.770672 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb249c-b5e4-46a0-8d28-690bc2fe3b07 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:32.771101 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb249c-b5e4-46a0-8d28-690bc2fe3b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:32.771377 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb249c-b5e4-46a0-8d28-690bc2fe3b07 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:32.795242 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb249c-b5e4-46a0-8d28-690bc2fe3b07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:32.798243 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44dc1a7-2294-4541-be4d-ec6ff82e2726 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:32.804061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-11fb249c-b5e4-46a0-8d28-690bc2fe3b07 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:32.804061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:32.804061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:32.804061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:32.804061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:32.804061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:32.804061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:32.804061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:32.804061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:32.804061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:32.804061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:32.804061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:32.804061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:32.804061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:32.804061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:32.804061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:32.805427 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:32.805427 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:32.805427 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:32.805427 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:32.805427 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:32.805427 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:32.805427 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:32.805427 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:32.805427 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:32.805427 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:32.805427 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:32.805427 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:32.805427 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:32.805427 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:32.805427 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:32.805427 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:32.806486 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:32.806486 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:32.806486 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:32.806486 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:32.806486 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:32.806486 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:32.806486 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:32.806486 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:32.806486 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:32.806486 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:32.806486 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:32.806486 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:32.807769 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 253/489] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 15:58:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:58:32.809177 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4708e9f0-abf8-498b-9402-a8a001746738 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:32.809534 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4708e9f0-abf8-498b-9402-a8a001746738 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:32.809877 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4708e9f0-abf8-498b-9402-a8a001746738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:32.832718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-4708e9f0-abf8-498b-9402-a8a001746738 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:32.835206 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-4708e9f0-abf8-498b-9402-a8a001746738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:32.843793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a9227-9acc-4d56-b071-d5e0a58f0902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:32.844550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a9227-9acc-4d56-b071-d5e0a58f0902 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:32.844945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a9227-9acc-4d56-b071-d5e0a58f0902 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:32.845315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a9227-9acc-4d56-b071-d5e0a58f0902 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:32.846458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a9227-9acc-4d56-b071-d5e0a58f0902 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:32.846458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a9227-9acc-4d56-b071-d5e0a58f0902 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:32.848037 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a9227-9acc-4d56-b071-d5e0a58f0902 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:32.862101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 237/490] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:32.863530 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a9227-9acc-4d56-b071-d5e0a58f0902 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:32.890957 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:32.897485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:32.934047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 254/491] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:32.935124 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4708e9f0-abf8-498b-9402-a8a001746738 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:32.995384 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a9227-9acc-4d56-b071-d5e0a58f0902 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:33.042724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f2712-7270-4e35-9c4d-3efdc41f55d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:33.043540 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f2712-7270-4e35-9c4d-3efdc41f55d8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:33.043901 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f2712-7270-4e35-9c4d-3efdc41f55d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:33.044222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f2712-7270-4e35-9c4d-3efdc41f55d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:33.045077 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f2712-7270-4e35-9c4d-3efdc41f55d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:33.045624 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f2712-7270-4e35-9c4d-3efdc41f55d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:33.047332 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f2712-7270-4e35-9c4d-3efdc41f55d8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:33.056764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 238/492] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 15:58:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:33.057812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f2712-7270-4e35-9c4d-3efdc41f55d8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:33.072823 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4708e9f0-abf8-498b-9402-a8a001746738 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:33.117645 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d42ba-8b87-4a8d-9de7-300c968a3d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:33.118618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d42ba-8b87-4a8d-9de7-300c968a3d62 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:33.118978 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d42ba-8b87-4a8d-9de7-300c968a3d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:33.119323 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d42ba-8b87-4a8d-9de7-300c968a3d62 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:33.127817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d42ba-8b87-4a8d-9de7-300c968a3d62 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:33.129306 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d42ba-8b87-4a8d-9de7-300c968a3d62 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:33.147178 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-9a2d42ba-8b87-4a8d-9de7-300c968a3d62 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 15:58:33.147178 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:33.147178 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:33.147178 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:33.147178 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:33.147178 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:33.147178 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:33.147178 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:33.147178 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:33.147178 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:33.147178 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:33.147178 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:33.147178 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:58:33.147178 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:58:33.147178 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:58:33.147178 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:58:33.148295 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:33.148295 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:33.148295 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:58:33.148295 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:33.148295 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:58:33.148295 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:33.148295 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:33.148295 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:33.148295 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:33.148295 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:33.148295 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:33.148295 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:33.148295 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:33.148295 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:33.148295 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:33.148295 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:33.148295 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:33.148295 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:33.149405 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:33.149405 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:33.149405 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:58:33.149405 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:58:33.149405 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:58:33.149405 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:58:33.149405 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:58:33.149405 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:58:33.149405 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 15:58:33.149405 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:33.150864 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 255/493] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jun 16 15:58:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:33.151886 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d42ba-8b87-4a8d-9de7-300c968a3d62 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:33.175827 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:33.190448 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:33.191523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f2712-7270-4e35-9c4d-3efdc41f55d8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:33.243354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a642d-9302-48dc-9290-181632df5dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:33.244419 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a642d-9302-48dc-9290-181632df5dec None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:33.244851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a642d-9302-48dc-9290-181632df5dec None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:33.245577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a642d-9302-48dc-9290-181632df5dec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:33.262253 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a642d-9302-48dc-9290-181632df5dec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:33.267853 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d42ba-8b87-4a8d-9de7-300c968a3d62 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:33.273411 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-2f7a642d-9302-48dc-9290-181632df5dec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:33.273411 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:33.273411 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:33.273411 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:33.273411 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:33.273411 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:33.273411 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:33.273411 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:33.273411 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:33.273411 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:33.273411 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:33.273411 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:33.273411 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:33.273411 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:33.273411 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:33.273411 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:33.274537 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:33.274537 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:33.274537 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:33.274537 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:33.274537 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:33.274537 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:33.274537 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:33.274537 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:33.274537 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:33.274537 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:33.274537 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:33.274537 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:33.274537 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:33.274537 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:33.274537 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:33.274537 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:33.275520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:33.275520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:33.275520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:33.275520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:33.275520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:33.275520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:33.275520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:33.275520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:33.275520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:33.275520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:33.275520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:33.275520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:33.277940 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 239/494] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 15:58:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:33.279053 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a642d-9302-48dc-9290-181632df5dec None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:33.313830 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d65bd0-7568-4dd5-80d4-0eec5ac5eec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:33.314880 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d65bd0-7568-4dd5-80d4-0eec5ac5eec0 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:33.315269 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d65bd0-7568-4dd5-80d4-0eec5ac5eec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:33.315691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d65bd0-7568-4dd5-80d4-0eec5ac5eec0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:33.329731 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d65bd0-7568-4dd5-80d4-0eec5ac5eec0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:33.331563 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d65bd0-7568-4dd5-80d4-0eec5ac5eec0 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:33.353802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-50d65bd0-7568-4dd5-80d4-0eec5ac5eec0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:33.353802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:33.353802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:33.353802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:33.353802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:33.353802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:33.353802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:33.353802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:33.353802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:33.353802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:33.353802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:33.353802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:33.353802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:33.353802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:33.353802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:33.355045 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:33.355045 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:33.355045 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:33.355045 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:33.355045 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:33.355045 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:33.355045 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:33.355045 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:33.355045 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:33.355045 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:33.355045 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:33.355045 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:33.355045 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:33.355045 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:33.355045 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:33.355045 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:33.355045 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:33.356106 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:33.356106 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:33.356106 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:33.356106 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:33.357716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 256/495] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:33.358851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d65bd0-7568-4dd5-80d4-0eec5ac5eec0 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:33.418457 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a642d-9302-48dc-9290-181632df5dec None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:33.456836 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41822c7d-f37d-4eae-912f-d9101519bb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:33.457527 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41822c7d-f37d-4eae-912f-d9101519bb94 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:33.457816 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41822c7d-f37d-4eae-912f-d9101519bb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:33.458242 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41822c7d-f37d-4eae-912f-d9101519bb94 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:33.459170 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41822c7d-f37d-4eae-912f-d9101519bb94 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:33.459342 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41822c7d-f37d-4eae-912f-d9101519bb94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:33.459445 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:33.461095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41822c7d-f37d-4eae-912f-d9101519bb94 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:33.478498 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d65bd0-7568-4dd5-80d4-0eec5ac5eec0 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:33.501954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 240/496] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jun 16 15:58:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:33.502816 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41822c7d-f37d-4eae-912f-d9101519bb94 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:33.514742 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:33.523237 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da823865-67eb-40ee-82af-d9eef76255b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:33.523918 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da823865-67eb-40ee-82af-d9eef76255b1 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:33.524200 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da823865-67eb-40ee-82af-d9eef76255b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:33.524483 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da823865-67eb-40ee-82af-d9eef76255b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:33.525425 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da823865-67eb-40ee-82af-d9eef76255b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:33.525555 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da823865-67eb-40ee-82af-d9eef76255b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:33.526995 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da823865-67eb-40ee-82af-d9eef76255b1 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:33.535522 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 257/497] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 15:58:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:33.536855 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b22a8b-76e9-4b5f-9bf6-43f1daf25a90 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:33.537148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b22a8b-76e9-4b5f-9bf6-43f1daf25a90 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:33.537425 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b22a8b-76e9-4b5f-9bf6-43f1daf25a90 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:33.539023 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 258/498] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:33.539779 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b22a8b-76e9-4b5f-9bf6-43f1daf25a90 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:33.616215 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41822c7d-f37d-4eae-912f-d9101519bb94 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:33.643220 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b22a8b-76e9-4b5f-9bf6-43f1daf25a90 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:33.651399 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0d857-f6fb-44ef-8b8b-fef5b1e780bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:33.652065 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0d857-f6fb-44ef-8b8b-fef5b1e780bb None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:33.652352 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0d857-f6fb-44ef-8b8b-fef5b1e780bb None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:33.652631 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0d857-f6fb-44ef-8b8b-fef5b1e780bb None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:33.653362 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce0d857-f6fb-44ef-8b8b-fef5b1e780bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:33.653671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce0d857-f6fb-44ef-8b8b-fef5b1e780bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:33.655216 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce0d857-f6fb-44ef-8b8b-fef5b1e780bb None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:33.666860 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 241/499] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:33.668155 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a884d80-549d-47a0-87d6-378522a68cec None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:33.668448 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a884d80-549d-47a0-87d6-378522a68cec None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:33.668810 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a884d80-549d-47a0-87d6-378522a68cec None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:33.670788 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 242/500] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:33.672637 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a884d80-549d-47a0-87d6-378522a68cec None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:33.681125 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b6167-691c-4018-987e-f65e4cf0bee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:33.681771 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b6167-691c-4018-987e-f65e4cf0bee4 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:33.682088 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b6167-691c-4018-987e-f65e4cf0bee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:33.682359 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b6167-691c-4018-987e-f65e4cf0bee4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:33.702496 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:33.744736 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:33.755426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763b6167-691c-4018-987e-f65e4cf0bee4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:33.757031 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763b6167-691c-4018-987e-f65e4cf0bee4 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:33.780657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a884d80-549d-47a0-87d6-378522a68cec None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:33.816470 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca22e7-93e7-4f29-9a98-d2df5943278d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:33.817358 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca22e7-93e7-4f29-9a98-d2df5943278d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:33.817646 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca22e7-93e7-4f29-9a98-d2df5943278d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:33.817936 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca22e7-93e7-4f29-9a98-d2df5943278d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:33.828087 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca22e7-93e7-4f29-9a98-d2df5943278d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:33.829556 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca22e7-93e7-4f29-9a98-d2df5943278d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:33.848865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-9dca22e7-93e7-4f29-9a98-d2df5943278d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:33.848865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:33.848865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:33.848865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:33.848865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:33.848865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:33.848865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:33.848865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:33.848865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:33.848865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:33.848865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:33.848865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:33.848865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:33.848865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:33.848865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:33.849724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:33.849724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:33.849724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:33.849724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:33.849724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:33.849724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:33.849724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:33.849724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:33.849724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:33.849724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:33.849724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:33.849724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:33.849724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:33.849724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:33.849724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:33.849724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:33.849724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:33.851060 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:33.851060 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:33.851060 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:33.851060 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:33.851060 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 243/501] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:33.851543 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae34193-2c02-4ee8-9b41-3fe976b00d0a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:33.851543 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae34193-2c02-4ee8-9b41-3fe976b00d0a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:33.851812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae34193-2c02-4ee8-9b41-3fe976b00d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:33.861622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 259/502] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:33.862507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b6167-691c-4018-987e-f65e4cf0bee4 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:33.869234 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-2ae34193-2c02-4ee8-9b41-3fe976b00d0a None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:33.870489 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2ae34193-2c02-4ee8-9b41-3fe976b00d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:33.912927 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:33.939776 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 244/503] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:33.941145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f7bd1-92fe-4390-8d25-f7331b724d6d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:33.941451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f7bd1-92fe-4390-8d25-f7331b724d6d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:33.941729 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f7bd1-92fe-4390-8d25-f7331b724d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:33.966221 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-3f8f7bd1-92fe-4390-8d25-f7331b724d6d None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:33.967924 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b6167-691c-4018-987e-f65e4cf0bee4 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:33.968801 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3f8f7bd1-92fe-4390-8d25-f7331b724d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:34.004576 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1328bbf7-e771-4180-8490-330d558a3c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:34.005523 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1328bbf7-e771-4180-8490-330d558a3c21 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:34.005965 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1328bbf7-e771-4180-8490-330d558a3c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:34.006345 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1328bbf7-e771-4180-8490-330d558a3c21 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:34.016053 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1328bbf7-e771-4180-8490-330d558a3c21 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:34.017887 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1328bbf7-e771-4180-8490-330d558a3c21 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:34.020683 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:34.034365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-1328bbf7-e771-4180-8490-330d558a3c21 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 15:58:34.034365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:34.034365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:34.034365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:34.034365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:34.034365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:34.034365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:34.034365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:34.034365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:34.034365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:34.034365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:34.034365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:34.034365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:58:34.034365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:58:34.034365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:58:34.034365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:58:34.035477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:34.035477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:34.035477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:58:34.035477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:34.035477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:58:34.035477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:34.035477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:34.035477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:34.035477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:34.035477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:34.035477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:34.035477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:34.035477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:34.035477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:34.035477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:34.035477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:34.035477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:34.035477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:34.036356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:34.036356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:34.036356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:58:34.036356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:58:34.036356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:58:34.036356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:58:34.036356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:58:34.036356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:58:34.036356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 15:58:34.036356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:34.036356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 260/504] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:34.036356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1328bbf7-e771-4180-8490-330d558a3c21 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:34.048641 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 245/505] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:34.049505 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f7bd1-92fe-4390-8d25-f7331b724d6d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:34.143613 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1328bbf7-e771-4180-8490-330d558a3c21 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:34.150558 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f7bd1-92fe-4390-8d25-f7331b724d6d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:34.154006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:34.181188 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde2d2a-097b-4bf5-a5c0-d76a56468d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:34.181856 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde2d2a-097b-4bf5-a5c0-d76a56468d93 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:34.182166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde2d2a-097b-4bf5-a5c0-d76a56468d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:34.182446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde2d2a-097b-4bf5-a5c0-d76a56468d93 None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:34.183163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde2d2a-097b-4bf5-a5c0-d76a56468d93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:34.183470 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde2d2a-097b-4bf5-a5c0-d76a56468d93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:34.184862 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde2d2a-097b-4bf5-a5c0-d76a56468d93 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:34.186433 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319cbd7b-02f9-4bbc-811f-0321457658fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:34.187259 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319cbd7b-02f9-4bbc-811f-0321457658fb None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:34.187544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319cbd7b-02f9-4bbc-811f-0321457658fb None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:34.187822 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319cbd7b-02f9-4bbc-811f-0321457658fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:34.197873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 261/506] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:34.198034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319cbd7b-02f9-4bbc-811f-0321457658fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:34.198778 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde2d2a-097b-4bf5-a5c0-d76a56468d93 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:34.199549 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319cbd7b-02f9-4bbc-811f-0321457658fb None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:34.221808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-319cbd7b-02f9-4bbc-811f-0321457658fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:34.221808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:34.221808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:34.221808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:34.221808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:34.221808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:34.221808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:34.221808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:34.221808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:34.221808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:34.221808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:34.221808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:34.221808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:34.221808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:34.221808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:34.222755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:34.222755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:34.222755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:34.222755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:34.222755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:34.222755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:34.222755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:34.222755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:34.222755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:34.222755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:34.222755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:34.222755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:34.222755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:34.222755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:34.222755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:34.222755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:34.222755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:34.223586 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:34.223586 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:34.223586 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:34.223586 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:34.223586 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 246/507] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:34.223840 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319cbd7b-02f9-4bbc-811f-0321457658fb None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:34.255734 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:34.313054 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde2d2a-097b-4bf5-a5c0-d76a56468d93 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:34.338735 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319cbd7b-02f9-4bbc-811f-0321457658fb None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:34.349343 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7900e6a-a801-4172-af4b-a9655d840d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:34.350177 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7900e6a-a801-4172-af4b-a9655d840d4a None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:34.350535 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7900e6a-a801-4172-af4b-a9655d840d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:34.350854 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7900e6a-a801-4172-af4b-a9655d840d4a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:34.361692 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7900e6a-a801-4172-af4b-a9655d840d4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:34.369426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-e7900e6a-a801-4172-af4b-a9655d840d4a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:34.369426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:34.369426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:34.369426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:34.369426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:34.369426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:34.369426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:34.369426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:34.369426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:34.369426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:34.369426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:34.369426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:34.369426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:34.369426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:34.369426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:34.369426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:34.370355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:34.370355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:34.370355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:34.370355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:34.370355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:34.370355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:34.370355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:34.370355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:34.370355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:34.370355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:34.370355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:34.370355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:34.370355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:34.370355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:34.370355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:34.370355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:34.371145 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:34.371145 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:34.371145 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:34.371145 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:34.371145 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:34.371145 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:34.371145 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:34.371145 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:34.371145 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:34.371145 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:34.371145 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:34.371145 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:34.371145 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 262/508] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 15:58:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:34.371824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7900e6a-a801-4172-af4b-a9655d840d4a None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:34.379125 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4cdf5-4aa5-4372-a89b-5bd00b8194bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:34.379810 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4cdf5-4aa5-4372-a89b-5bd00b8194bb None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:34.380097 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4cdf5-4aa5-4372-a89b-5bd00b8194bb None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:34.380372 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4cdf5-4aa5-4372-a89b-5bd00b8194bb None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:34.381375 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4cdf5-4aa5-4372-a89b-5bd00b8194bb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:34.381611 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4cdf5-4aa5-4372-a89b-5bd00b8194bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:34.383018 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4cdf5-4aa5-4372-a89b-5bd00b8194bb None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:34.401420 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 247/509] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jun 16 15:58:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:34.401620 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:34.402347 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4cdf5-4aa5-4372-a89b-5bd00b8194bb None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:34.479105 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7900e6a-a801-4172-af4b-a9655d840d4a None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:34.496304 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:34.507700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4cdf5-4aa5-4372-a89b-5bd00b8194bb None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:34.515334 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a8176-5214-4fcf-811c-044fda9b2971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:34.516118 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a8176-5214-4fcf-811c-044fda9b2971 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:34.516408 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a8176-5214-4fcf-811c-044fda9b2971 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:34.516679 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a8176-5214-4fcf-811c-044fda9b2971 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:34.527806 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a8176-5214-4fcf-811c-044fda9b2971 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:34.529284 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a8176-5214-4fcf-811c-044fda9b2971 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:34.544105 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334724f-6a66-4dcf-a9f7-56bce49beb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:34.544956 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334724f-6a66-4dcf-a9f7-56bce49beb3f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:34.545780 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334724f-6a66-4dcf-a9f7-56bce49beb3f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:34.546089 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334724f-6a66-4dcf-a9f7-56bce49beb3f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:34.547406 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-777a8176-5214-4fcf-811c-044fda9b2971 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:34.547406 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:34.547406 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:34.547406 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:34.547406 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:34.547406 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:34.547406 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:34.547406 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:34.547406 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:34.547406 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:34.547406 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:34.547406 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:34.547406 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:34.547406 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:34.547406 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:34.548263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:34.548263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:34.548263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:34.548263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:34.548263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:34.548263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:34.548263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:34.548263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:34.548263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:34.548263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:34.548263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:34.548263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:34.548263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:34.548263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:34.548263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:34.548263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:34.548263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:34.549098 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:34.549098 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:34.549098 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:34.549098 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:34.549098 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 263/510] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:34.549869 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a8176-5214-4fcf-811c-044fda9b2971 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:34.554796 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334724f-6a66-4dcf-a9f7-56bce49beb3f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:34.556272 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334724f-6a66-4dcf-a9f7-56bce49beb3f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:34.573300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-9334724f-6a66-4dcf-a9f7-56bce49beb3f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 15:58:34.573300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:34.573300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:34.573300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:34.573300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:34.573300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:34.573300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:34.573300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:34.573300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:34.573300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:34.573300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:34.573300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:34.573300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:58:34.573300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:58:34.573300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:58:34.573300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:58:34.574485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:34.574485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:34.574485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:58:34.574485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:34.574485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:58:34.574485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:34.574485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:34.574485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:34.574485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:34.574485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:34.574485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:34.574485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:34.574485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:34.574485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:34.574485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:34.574485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:34.574485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:34.574485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:34.575373 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:34.575373 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:34.575373 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:58:34.575373 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:58:34.575373 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:58:34.575373 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:58:34.575373 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:58:34.575373 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:58:34.575373 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 15:58:34.575373 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:34.575373 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 248/511] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:34.575373 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334724f-6a66-4dcf-a9f7-56bce49beb3f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:34.644041 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:34.663698 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a8176-5214-4fcf-811c-044fda9b2971 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:34.681990 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334724f-6a66-4dcf-a9f7-56bce49beb3f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:34.698387 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e17ace-f1a2-4819-bbfe-217766371c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:34.699053 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e17ace-f1a2-4819-bbfe-217766371c6e None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:34.699338 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e17ace-f1a2-4819-bbfe-217766371c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:34.699609 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e17ace-f1a2-4819-bbfe-217766371c6e None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:34.700328 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e17ace-f1a2-4819-bbfe-217766371c6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:34.700633 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e17ace-f1a2-4819-bbfe-217766371c6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:34.703308 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e17ace-f1a2-4819-bbfe-217766371c6e None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:34.715028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 264/512] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:34.715832 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e17ace-f1a2-4819-bbfe-217766371c6e None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:34.720143 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83155547-4180-4673-bb5d-300f5728b26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:34.720818 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83155547-4180-4673-bb5d-300f5728b26d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:34.721100 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83155547-4180-4673-bb5d-300f5728b26d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:34.721377 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83155547-4180-4673-bb5d-300f5728b26d None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:34.722157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83155547-4180-4673-bb5d-300f5728b26d None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:34.722466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83155547-4180-4673-bb5d-300f5728b26d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:34.723855 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83155547-4180-4673-bb5d-300f5728b26d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:34.733261 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 249/513] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 15:58:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:34.733527 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83155547-4180-4673-bb5d-300f5728b26d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:34.749582 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:34.824371 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e17ace-f1a2-4819-bbfe-217766371c6e None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:34.837355 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83155547-4180-4673-bb5d-300f5728b26d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:34.859755 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a29fc-07e3-4549-a258-240d7198f486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:34.860411 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a29fc-07e3-4549-a258-240d7198f486 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:34.860695 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a29fc-07e3-4549-a258-240d7198f486 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:34.860970 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a29fc-07e3-4549-a258-240d7198f486 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:34.871873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:34.872870 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22d650-9dd7-458c-bf06-f51bbfd2cd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:34.873828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22d650-9dd7-458c-bf06-f51bbfd2cd10 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:34.874213 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22d650-9dd7-458c-bf06-f51bbfd2cd10 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:34.874848 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22d650-9dd7-458c-bf06-f51bbfd2cd10 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:34.875400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec22d650-9dd7-458c-bf06-f51bbfd2cd10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:34.875711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec22d650-9dd7-458c-bf06-f51bbfd2cd10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:34.877106 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec22d650-9dd7-458c-bf06-f51bbfd2cd10 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:34.890666 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 250/514] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:34.891525 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22d650-9dd7-458c-bf06-f51bbfd2cd10 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:34.940938 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288a29fc-07e3-4549-a258-240d7198f486 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:34.942542 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288a29fc-07e3-4549-a258-240d7198f486 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:34.971157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:35.002388 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22d650-9dd7-458c-bf06-f51bbfd2cd10 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:35.037803 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03f912-19a2-482a-85fc-b5c3d17ccfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:35.038549 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03f912-19a2-482a-85fc-b5c3d17ccfb2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:35.038796 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03f912-19a2-482a-85fc-b5c3d17ccfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:35.039074 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03f912-19a2-482a-85fc-b5c3d17ccfb2 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:35.039918 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb03f912-19a2-482a-85fc-b5c3d17ccfb2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:35.040224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb03f912-19a2-482a-85fc-b5c3d17ccfb2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:35.041695 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb03f912-19a2-482a-85fc-b5c3d17ccfb2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:35.050456 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 265/515] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:35.051434 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a29fc-07e3-4549-a258-240d7198f486 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:35.061304 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 251/516] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jun 16 15:58:34 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:35.062187 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03f912-19a2-482a-85fc-b5c3d17ccfb2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:35.099643 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:35.159044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a29fc-07e3-4549-a258-240d7198f486 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:35.165659 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03f912-19a2-482a-85fc-b5c3d17ccfb2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:35.193752 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c60d28-0083-42dd-892f-e39747d22ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:35.194706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c60d28-0083-42dd-892f-e39747d22ced None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:35.194995 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c60d28-0083-42dd-892f-e39747d22ced None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:35.195272 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c60d28-0083-42dd-892f-e39747d22ced None admin] PATH_INFO: `/v3/projects/service` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:35.195668 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:35.202905 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a960af-b2ed-4de0-b10b-2e5aeea0e707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:35.203209 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c60d28-0083-42dd-892f-e39747d22ced None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:35.203859 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a960af-b2ed-4de0-b10b-2e5aeea0e707 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:35.204159 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a960af-b2ed-4de0-b10b-2e5aeea0e707 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:35.204434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a960af-b2ed-4de0-b10b-2e5aeea0e707 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:35.204686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c60d28-0083-42dd-892f-e39747d22ced None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:35.214837 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a960af-b2ed-4de0-b10b-2e5aeea0e707 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:35.216277 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a960af-b2ed-4de0-b10b-2e5aeea0e707 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:35.219974 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-58c60d28-0083-42dd-892f-e39747d22ced None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:35.219974 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:35.219974 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:35.219974 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:35.219974 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:35.219974 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:35.219974 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:35.219974 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:35.219974 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:35.219974 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:35.219974 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:35.219974 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:35.219974 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:58:35.219974 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:58:35.219974 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:58:35.221006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:58:35.221006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:35.221006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:35.221006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:35.221006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:35.221006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:35.221006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:35.221006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:35.221006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:35.221006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:35.221006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:35.221006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:35.221006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:35.221006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:35.221006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:35.221006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:35.221006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:35.222077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:58:35.222077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:58:35.222077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:58:35.222077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:58:35.222077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:58:35.222077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:58:35.222077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:35.222077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:35.222077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 266/517] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:35.223483 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c60d28-0083-42dd-892f-e39747d22ced None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:35.236273 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-f4a960af-b2ed-4de0-b10b-2e5aeea0e707 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:35.236273 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:35.236273 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:35.236273 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:35.236273 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:35.236273 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:35.236273 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:35.236273 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:35.236273 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:35.236273 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:35.236273 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:35.236273 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:35.236273 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:35.236273 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:35.236273 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:35.237313 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:35.237313 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:35.237313 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:35.237313 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:35.237313 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:35.237313 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:35.237313 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:35.237313 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:35.237313 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:35.237313 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:35.237313 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:35.237313 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:35.237313 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:35.237313 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:35.237313 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:35.237313 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:35.237313 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:35.238381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:35.238381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:35.238381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:35.238381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:35.242367 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 252/518] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:35.243011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a960af-b2ed-4de0-b10b-2e5aeea0e707 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:35.334217 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:35.337536 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c60d28-0083-42dd-892f-e39747d22ced None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:35.352680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a960af-b2ed-4de0-b10b-2e5aeea0e707 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:35.372637 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9480fa89-a680-4135-8726-5eaf7edfc499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:35.373836 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480fa89-a680-4135-8726-5eaf7edfc499 None admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:35.374152 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480fa89-a680-4135-8726-5eaf7edfc499 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:35.374447 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480fa89-a680-4135-8726-5eaf7edfc499 None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:35.375144 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9480fa89-a680-4135-8726-5eaf7edfc499 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:35.376626 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9480fa89-a680-4135-8726-5eaf7edfc499 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:35.395376 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327b082-1fab-402c-a6b5-4ca18afd81dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:35.396176 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327b082-1fab-402c-a6b5-4ca18afd81dc None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:35.396456 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327b082-1fab-402c-a6b5-4ca18afd81dc None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:35.396734 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327b082-1fab-402c-a6b5-4ca18afd81dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:35.397815 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 267/519] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 15:58:35 2022] POST /identity/v3/users => generated 247 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:58:35.398688 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9480fa89-a680-4135-8726-5eaf7edfc499 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:35.421042 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327b082-1fab-402c-a6b5-4ca18afd81dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:35.422780 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327b082-1fab-402c-a6b5-4ca18afd81dc None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:35.449209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-c327b082-1fab-402c-a6b5-4ca18afd81dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:35.449209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:35.449209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:35.449209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:35.449209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:35.449209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:35.449209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:35.449209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:35.449209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:35.449209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:35.449209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:35.449209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:35.449209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:35.449209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:35.449209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:35.450369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:35.450369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:35.450369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:35.450369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:35.450369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:35.450369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:35.450369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:35.450369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:35.450369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:35.450369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:35.450369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:35.450369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:35.450369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:35.450369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:35.450369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:35.450369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:35.450369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:35.452190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:35.452190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:35.452190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:35.452190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:35.452190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 253/520] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:35.453444 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327b082-1fab-402c-a6b5-4ca18afd81dc None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:35.492631 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:35.526587 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9480fa89-a680-4135-8726-5eaf7edfc499 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:35.568733 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0bc4c-0c10-4bbf-8d9c-13b99ed29397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:35.569399 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0bc4c-0c10-4bbf-8d9c-13b99ed29397 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:35.569690 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0bc4c-0c10-4bbf-8d9c-13b99ed29397 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:35.570005 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0bc4c-0c10-4bbf-8d9c-13b99ed29397 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:35.572152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327b082-1fab-402c-a6b5-4ca18afd81dc None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:35.604197 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:35.618051 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749c7b43-696e-4f57-a251-5095ac5f4f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:35.618848 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c7b43-696e-4f57-a251-5095ac5f4f6b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:35.619128 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c7b43-696e-4f57-a251-5095ac5f4f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:35.619410 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c7b43-696e-4f57-a251-5095ac5f4f6b None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:35.620209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749c7b43-696e-4f57-a251-5095ac5f4f6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:35.620545 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749c7b43-696e-4f57-a251-5095ac5f4f6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:35.622088 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749c7b43-696e-4f57-a251-5095ac5f4f6b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:35.642599 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 254/521] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:58:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:35.643455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749c7b43-696e-4f57-a251-5095ac5f4f6b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:35.661168 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0bc4c-0c10-4bbf-8d9c-13b99ed29397 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:35.662759 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0bc4c-0c10-4bbf-8d9c-13b99ed29397 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:35.747585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:35.755515 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749c7b43-696e-4f57-a251-5095ac5f4f6b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:35.793764 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 268/522] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:35.793764 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0bc4c-0c10-4bbf-8d9c-13b99ed29397 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:35.806362 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e525ce8-76d8-4da3-a892-18b33354e047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:35.807350 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e525ce8-76d8-4da3-a892-18b33354e047 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:35.807776 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e525ce8-76d8-4da3-a892-18b33354e047 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:35.808178 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e525ce8-76d8-4da3-a892-18b33354e047 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:35.808904 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e525ce8-76d8-4da3-a892-18b33354e047 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:35.809333 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e525ce8-76d8-4da3-a892-18b33354e047 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:35.812153 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e525ce8-76d8-4da3-a892-18b33354e047 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:35.832562 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 255/523] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:35.832562 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e525ce8-76d8-4da3-a892-18b33354e047 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:35.856358 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:35.902762 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0bc4c-0c10-4bbf-8d9c-13b99ed29397 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:35.940083 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24eb466-53c6-473b-bd61-649e9ac4edbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:35.941133 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24eb466-53c6-473b-bd61-649e9ac4edbc None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:35.941679 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24eb466-53c6-473b-bd61-649e9ac4edbc None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:35.941981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24eb466-53c6-473b-bd61-649e9ac4edbc None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:35.942749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24eb466-53c6-473b-bd61-649e9ac4edbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:35.943060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24eb466-53c6-473b-bd61-649e9ac4edbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:35.944575 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24eb466-53c6-473b-bd61-649e9ac4edbc None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:35.958442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 269/524] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:35.959297 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24eb466-53c6-473b-bd61-649e9ac4edbc None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:35.965459 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e525ce8-76d8-4da3-a892-18b33354e047 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:36.007467 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ee7b6-3c1d-4273-b1a8-a3cde5f4f98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:36.008371 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ee7b6-3c1d-4273-b1a8-a3cde5f4f98d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:36.008658 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ee7b6-3c1d-4273-b1a8-a3cde5f4f98d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:36.009324 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ee7b6-3c1d-4273-b1a8-a3cde5f4f98d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:36.010209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ee7b6-3c1d-4273-b1a8-a3cde5f4f98d None admin] RBAC: Adding query filter params (role.id=cfc67444e9944ffc87173be8c9245108, scope.project.id=b1177a85a0de4276898daf4a24ff069f, user.id=c3645b1a0be34c01ac87c734fcd85ff6) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:36.010525 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ee7b6-3c1d-4273-b1a8-a3cde5f4f98d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:36.012655 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ee7b6-3c1d-4273-b1a8-a3cde5f4f98d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:36.030313 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 256/525] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:58:35 2022] GET /identity/v3/role_assignments?user.id=c3645b1a0be34c01ac87c734fcd85ff6&scope.project.id=b1177a85a0de4276898daf4a24ff069f&role.id=cfc67444e9944ffc87173be8c9245108 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:36.031704 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ee7b6-3c1d-4273-b1a8-a3cde5f4f98d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:36.058993 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:36.086471 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24eb466-53c6-473b-bd61-649e9ac4edbc None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:36.105358 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:36.129492 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b30c5-d9d1-4d81-ac50-2592825ef4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:36.130426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b30c5-d9d1-4d81-ac50-2592825ef4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:36.130813 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b30c5-d9d1-4d81-ac50-2592825ef4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:36.131366 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b30c5-d9d1-4d81-ac50-2592825ef4b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:36.145353 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b30c5-d9d1-4d81-ac50-2592825ef4b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:36.147066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b30c5-d9d1-4d81-ac50-2592825ef4b8 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:36.163751 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ee7b6-3c1d-4273-b1a8-a3cde5f4f98d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:36.168778 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-ce1b30c5-d9d1-4d81-ac50-2592825ef4b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:36.168778 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:36.168778 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:36.168778 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:36.168778 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:36.168778 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:36.168778 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:36.168778 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:36.168778 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:36.168778 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:36.168778 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:36.168778 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:36.168778 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:36.168778 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:36.168778 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:36.170131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:36.170131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:36.170131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:36.170131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:36.170131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:36.170131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:36.170131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:36.170131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:36.170131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:36.170131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:36.170131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:36.170131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:36.170131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:36.170131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:36.170131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:36.170131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:36.170131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:36.171136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:36.171136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:36.171136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:36.171136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:36.172768 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 270/526] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:36.173827 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b30c5-d9d1-4d81-ac50-2592825ef4b8 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:36.208238 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc2953-42bd-4f09-9928-7be866254aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:36.209492 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc2953-42bd-4f09-9928-7be866254aba None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:36.209492 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc2953-42bd-4f09-9928-7be866254aba None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:36.209698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc2953-42bd-4f09-9928-7be866254aba None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:36.219342 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dc2953-42bd-4f09-9928-7be866254aba None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:36.220811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dc2953-42bd-4f09-9928-7be866254aba None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:36.236275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-69dc2953-42bd-4f09-9928-7be866254aba None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 15:58:36.236275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:36.236275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:36.236275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:36.236275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:36.236275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:36.236275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:36.236275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:36.236275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:36.236275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:36.236275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:36.236275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:36.236275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:58:36.236275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:58:36.236275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:58:36.236275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:58:36.237357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:36.237357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:36.237357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:58:36.237357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:36.237357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:58:36.237357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:36.237357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:36.237357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:36.237357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:36.237357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:36.237357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:36.237357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:36.237357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:36.237357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:36.237357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:36.237357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:36.237357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:36.237357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:36.238447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:36.238447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:36.238447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:58:36.238447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:58:36.238447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:58:36.238447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:58:36.238447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:58:36.238447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:58:36.238447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 15:58:36.238447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:36.238447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 257/527] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 15:58:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:36.238447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc2953-42bd-4f09-9928-7be866254aba None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:36.299287 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b30c5-d9d1-4d81-ac50-2592825ef4b8 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:36.336252 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b08bf-fff6-4ef6-824a-a4a9a73a9de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:36.337218 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b08bf-fff6-4ef6-824a-a4a9a73a9de5 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:36.337604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b08bf-fff6-4ef6-824a-a4a9a73a9de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:36.338153 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b08bf-fff6-4ef6-824a-a4a9a73a9de5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:36.343524 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:36.349479 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27b08bf-fff6-4ef6-824a-a4a9a73a9de5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:36.351552 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27b08bf-fff6-4ef6-824a-a4a9a73a9de5 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:36.367779 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc2953-42bd-4f09-9928-7be866254aba None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:36.368235 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-c27b08bf-fff6-4ef6-824a-a4a9a73a9de5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:36.368235 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:36.368235 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:36.368235 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:36.368235 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:36.368235 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:36.368235 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:36.368235 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:36.368235 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:36.368235 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:36.368235 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:36.368235 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:36.368235 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:58:36.368235 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:58:36.368235 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:58:36.369073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:58:36.369073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:36.369073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:36.369073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:36.369073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:36.369073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:36.369073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:36.369073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:36.369073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:36.369073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:36.369073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:36.369073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:36.369073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:36.369073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:36.369073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:36.369073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:36.369073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:36.370317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:58:36.370317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:58:36.370317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:58:36.370317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:58:36.370317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:58:36.370317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:58:36.370317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:36.370317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:36.372822 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 271/528] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:36.374408 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b08bf-fff6-4ef6-824a-a4a9a73a9de5 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:36.409693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:36.421705 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404634f-2bd1-4bfd-b3fe-45757416cbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:36.422535 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404634f-2bd1-4bfd-b3fe-45757416cbf2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:36.422816 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404634f-2bd1-4bfd-b3fe-45757416cbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:36.423088 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404634f-2bd1-4bfd-b3fe-45757416cbf2 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:36.423810 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0404634f-2bd1-4bfd-b3fe-45757416cbf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:36.424111 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0404634f-2bd1-4bfd-b3fe-45757416cbf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:36.425485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0404634f-2bd1-4bfd-b3fe-45757416cbf2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:36.443250 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 258/529] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:36.444387 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404634f-2bd1-4bfd-b3fe-45757416cbf2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:36.496407 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b08bf-fff6-4ef6-824a-a4a9a73a9de5 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:36.535309 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9ba95-bc37-4765-a97c-031864eef8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:36.536535 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9ba95-bc37-4765-a97c-031864eef8cf None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:36.537025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9ba95-bc37-4765-a97c-031864eef8cf None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:36.537805 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9ba95-bc37-4765-a97c-031864eef8cf None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:36.539015 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac9ba95-bc37-4765-a97c-031864eef8cf None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:36.539536 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac9ba95-bc37-4765-a97c-031864eef8cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:36.541238 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac9ba95-bc37-4765-a97c-031864eef8cf None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:36.578904 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404634f-2bd1-4bfd-b3fe-45757416cbf2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:36.584998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 272/530] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jun 16 15:58:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:36.586401 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9ba95-bc37-4765-a97c-031864eef8cf None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:36.629325 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80132e73-2960-45a9-80ed-6e77c1e67e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:36.631146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80132e73-2960-45a9-80ed-6e77c1e67e97 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:36.631741 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80132e73-2960-45a9-80ed-6e77c1e67e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:36.632222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80132e73-2960-45a9-80ed-6e77c1e67e97 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:36.633969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80132e73-2960-45a9-80ed-6e77c1e67e97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:36.635430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80132e73-2960-45a9-80ed-6e77c1e67e97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:36.637111 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80132e73-2960-45a9-80ed-6e77c1e67e97 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:36.647379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:36.657021 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 259/531] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:58:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:36.658333 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80132e73-2960-45a9-80ed-6e77c1e67e97 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:36.695529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:36.713576 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9ba95-bc37-4765-a97c-031864eef8cf None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:36.753606 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2141837-98e7-4542-94f3-f49b2b78b4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:36.754588 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2141837-98e7-4542-94f3-f49b2b78b4da None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:36.754932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2141837-98e7-4542-94f3-f49b2b78b4da None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:36.755795 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2141837-98e7-4542-94f3-f49b2b78b4da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:36.764972 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2141837-98e7-4542-94f3-f49b2b78b4da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:36.767106 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2141837-98e7-4542-94f3-f49b2b78b4da None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:36.784010 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-f2141837-98e7-4542-94f3-f49b2b78b4da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:36.784010 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:36.784010 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:36.784010 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:36.784010 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:36.784010 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:36.784010 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:36.784010 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:36.784010 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:36.784010 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:36.784010 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:36.784010 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:36.784010 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:58:36.784010 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:58:36.784010 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:58:36.784919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:58:36.784919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:36.784919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:36.784919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:36.784919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:36.784919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:36.784919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:36.784919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:36.784919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:36.784919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:36.784919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:36.784919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:36.784919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:36.784919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:36.784919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:36.784919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:36.784919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:36.785777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:58:36.785777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:58:36.785777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:58:36.785777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:58:36.785777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:58:36.785777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:58:36.785777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:36.785777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:36.785777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 273/532] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:36.787038 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2141837-98e7-4542-94f3-f49b2b78b4da None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:36.805101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80132e73-2960-45a9-80ed-6e77c1e67e97 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:36.851868 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d5c4f-5e58-42ba-9b55-e8f9ddb11fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:36.852869 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d5c4f-5e58-42ba-9b55-e8f9ddb11fbd None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:36.853334 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d5c4f-5e58-42ba-9b55-e8f9ddb11fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:36.853789 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d5c4f-5e58-42ba-9b55-e8f9ddb11fbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:36.915381 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2141837-98e7-4542-94f3-f49b2b78b4da None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:36.938575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:36.943345 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d5c4f-5e58-42ba-9b55-e8f9ddb11fbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:36.945412 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d5c4f-5e58-42ba-9b55-e8f9ddb11fbd None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:36.965281 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff7087-9a41-4e93-a6d1-a0b648409b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:36.966565 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff7087-9a41-4e93-a6d1-a0b648409b9d None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:36.967613 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff7087-9a41-4e93-a6d1-a0b648409b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:36.968071 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff7087-9a41-4e93-a6d1-a0b648409b9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:36.969095 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff7087-9a41-4e93-a6d1-a0b648409b9d None admin] RBAC: Adding query filter params (role.id=cfc67444e9944ffc87173be8c9245108, scope.project.id=b1177a85a0de4276898daf4a24ff069f, user.id=6f11beca9c5148eea4cc45f8dc89dbda) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:36.969558 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff7087-9a41-4e93-a6d1-a0b648409b9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:36.972078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff7087-9a41-4e93-a6d1-a0b648409b9d None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:36.986632 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 274/533] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:58:36 2022] GET /identity/v3/role_assignments?user.id=6f11beca9c5148eea4cc45f8dc89dbda&scope.project.id=b1177a85a0de4276898daf4a24ff069f&role.id=cfc67444e9944ffc87173be8c9245108 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:36.986632 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff7087-9a41-4e93-a6d1-a0b648409b9d None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:37.012387 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:37.135394 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 260/534] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:37.141430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d5c4f-5e58-42ba-9b55-e8f9ddb11fbd None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:37.171591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff7087-9a41-4e93-a6d1-a0b648409b9d None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:37.215438 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b6a1e-d65b-421e-be37-23288db6f060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:37.216141 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b6a1e-d65b-421e-be37-23288db6f060 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:37.216426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b6a1e-d65b-421e-be37-23288db6f060 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:37.216696 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b6a1e-d65b-421e-be37-23288db6f060 None admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:37.217601 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920b6a1e-d65b-421e-be37-23288db6f060 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:37.219348 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920b6a1e-d65b-421e-be37-23288db6f060 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:37.219744 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920b6a1e-d65b-421e-be37-23288db6f060 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:37.249728 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 275/535] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:58:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:37.252171 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b6a1e-d65b-421e-be37-23288db6f060 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:37.350868 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:37.356400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d5c4f-5e58-42ba-9b55-e8f9ddb11fbd None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:37.394433 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb36167-054e-4fa9-ad77-57bfcf56c95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:37.395243 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb36167-054e-4fa9-ad77-57bfcf56c95b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:37.395523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb36167-054e-4fa9-ad77-57bfcf56c95b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:37.395794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb36167-054e-4fa9-ad77-57bfcf56c95b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:37.404027 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:37.407470 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb36167-054e-4fa9-ad77-57bfcf56c95b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:37.409058 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb36167-054e-4fa9-ad77-57bfcf56c95b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:37.432550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-fdb36167-054e-4fa9-ad77-57bfcf56c95b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:37.432550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:37.432550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:37.432550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:37.432550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:37.432550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:37.432550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:37.432550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:37.432550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:37.432550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:37.432550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:37.432550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:37.432550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:37.432550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:37.432550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:37.433604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:37.433604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:37.433604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:37.433604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:37.433604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:37.433604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:37.433604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:37.433604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:37.433604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:37.433604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:37.433604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:37.433604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:37.433604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:37.433604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:37.433604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:37.433604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:37.433604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:37.434737 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:37.434737 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:37.434737 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:37.434737 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:37.434737 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 261/536] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:37.443272 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb36167-054e-4fa9-ad77-57bfcf56c95b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:37.466468 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b6a1e-d65b-421e-be37-23288db6f060 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:37.530497 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a86fdcb-18d0-42c6-9ac0-5ee7decb8ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:37.531165 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86fdcb-18d0-42c6-9ac0-5ee7decb8ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:37.531449 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86fdcb-18d0-42c6-9ac0-5ee7decb8ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:37.531727 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86fdcb-18d0-42c6-9ac0-5ee7decb8ba2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:37.532575 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a86fdcb-18d0-42c6-9ac0-5ee7decb8ba2 None admin] RBAC: Adding query filter params (role.id=cfc67444e9944ffc87173be8c9245108, scope.project.id=b1177a85a0de4276898daf4a24ff069f, user.id=69b669a5ba4e4294baeea369949e3e6f) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:37.532888 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a86fdcb-18d0-42c6-9ac0-5ee7decb8ba2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:37.534396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a86fdcb-18d0-42c6-9ac0-5ee7decb8ba2 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:37.550454 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 276/537] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:58:37 2022] GET /identity/v3/role_assignments?user.id=69b669a5ba4e4294baeea369949e3e6f&scope.project.id=b1177a85a0de4276898daf4a24ff069f&role.id=cfc67444e9944ffc87173be8c9245108 => generated 259 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:37.601859 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb36167-054e-4fa9-ad77-57bfcf56c95b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:37.659912 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9476a401-e761-4a15-a6db-bd748350f16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:37.660615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476a401-e761-4a15-a6db-bd748350f16f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:37.660903 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476a401-e761-4a15-a6db-bd748350f16f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:37.661182 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476a401-e761-4a15-a6db-bd748350f16f None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:37.662021 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9476a401-e761-4a15-a6db-bd748350f16f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:37.662333 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9476a401-e761-4a15-a6db-bd748350f16f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:37.663819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9476a401-e761-4a15-a6db-bd748350f16f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:37.683993 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 262/538] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:37.690797 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a86fdcb-18d0-42c6-9ac0-5ee7decb8ba2 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:37.773313 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b8830-68c0-49fa-a4e9-acedd3126453 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:37.773605 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b8830-68c0-49fa-a4e9-acedd3126453 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:37.773792 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b8830-68c0-49fa-a4e9-acedd3126453 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:37.775446 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 263/539] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:58:37.791490 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643db30d-5d15-4a12-9f25-7466c006acb2 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:37.791780 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643db30d-5d15-4a12-9f25-7466c006acb2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:37.792059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643db30d-5d15-4a12-9f25-7466c006acb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:37.824635 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-643db30d-5d15-4a12-9f25-7466c006acb2 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:37.825846 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-643db30d-5d15-4a12-9f25-7466c006acb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:37.830461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:37.847678 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a86fdcb-18d0-42c6-9ac0-5ee7decb8ba2 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:37.894040 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47285e16-277e-481e-8b1a-3dbf503624cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:37.894847 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47285e16-277e-481e-8b1a-3dbf503624cf None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:37.895127 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47285e16-277e-481e-8b1a-3dbf503624cf None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:37.895399 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47285e16-277e-481e-8b1a-3dbf503624cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:37.897500 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:37.905609 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47285e16-277e-481e-8b1a-3dbf503624cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:37.907308 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47285e16-277e-481e-8b1a-3dbf503624cf None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:37.925176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-47285e16-277e-481e-8b1a-3dbf503624cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:37.925176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:37.925176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:37.925176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:37.925176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:37.925176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:37.925176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:37.925176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:37.925176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:37.925176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:37.925176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:37.925176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:37.925176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:58:37.925176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:58:37.925176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:58:37.926590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:58:37.926590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:37.926590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:37.926590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:37.926590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:37.926590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:37.926590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:37.926590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:37.926590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:37.926590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:37.926590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:37.926590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:37.926590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:37.926590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:37.926590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:37.926590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:37.926590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:37.928047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:58:37.928047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:58:37.928047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:58:37.928047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:58:37.928047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:58:37.928047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:58:37.928047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:37.928047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:37.928047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 277/540] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:37.932690 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47285e16-277e-481e-8b1a-3dbf503624cf None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:37.945971 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 264/541] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:37.987444 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643db30d-5d15-4a12-9f25-7466c006acb2 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:38.057998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47285e16-277e-481e-8b1a-3dbf503624cf None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:38.107213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9541f0c-209c-4208-9fc9-ef5fb84f795c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:38.108269 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9541f0c-209c-4208-9fc9-ef5fb84f795c None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:38.108682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9541f0c-209c-4208-9fc9-ef5fb84f795c None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:38.109113 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9541f0c-209c-4208-9fc9-ef5fb84f795c None admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:38.110625 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9541f0c-209c-4208-9fc9-ef5fb84f795c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:38.111073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9541f0c-209c-4208-9fc9-ef5fb84f795c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:38.118074 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9541f0c-209c-4208-9fc9-ef5fb84f795c None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:38.127465 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 278/542] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:58:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:38.133065 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9541f0c-209c-4208-9fc9-ef5fb84f795c None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:38.152131 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643db30d-5d15-4a12-9f25-7466c006acb2 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:38.195266 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:38.227235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c0bae-2d4e-47cf-b757-688686a76e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:38.228057 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c0bae-2d4e-47cf-b757-688686a76e8c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:38.228342 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c0bae-2d4e-47cf-b757-688686a76e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:38.228620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c0bae-2d4e-47cf-b757-688686a76e8c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:38.243409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9541f0c-209c-4208-9fc9-ef5fb84f795c None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:38.248723 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:38.253431 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27c0bae-2d4e-47cf-b757-688686a76e8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:38.274727 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-f27c0bae-2d4e-47cf-b757-688686a76e8c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:38.274727 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:38.274727 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:38.274727 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:38.274727 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:38.274727 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:38.274727 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:38.274727 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:38.274727 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:38.274727 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:38.274727 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:38.274727 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:38.274727 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:38.274727 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:38.274727 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:38.274727 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:38.275717 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:38.275717 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:38.275717 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:38.275717 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:38.275717 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:38.275717 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:38.275717 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:38.275717 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:38.275717 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:38.275717 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:38.275717 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:38.275717 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:38.275717 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:38.275717 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:38.275717 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:38.275717 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:38.276541 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:38.276541 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:38.276541 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:38.276541 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:38.276541 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:38.276541 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:38.276541 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:38.276541 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:38.276541 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:38.276541 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:38.276541 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:38.276541 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:38.276541 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 265/543] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 15:58:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:58:38.286953 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c0bae-2d4e-47cf-b757-688686a76e8c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:38.287245 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693a760-e01e-4083-9721-0dbc82b00001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:38.287915 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693a760-e01e-4083-9721-0dbc82b00001 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:38.288228 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693a760-e01e-4083-9721-0dbc82b00001 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:38.288477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693a760-e01e-4083-9721-0dbc82b00001 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:38.289308 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f693a760-e01e-4083-9721-0dbc82b00001 None admin] RBAC: Adding query filter params (role.id=cfc67444e9944ffc87173be8c9245108, scope.project.id=b1177a85a0de4276898daf4a24ff069f, user.id=5a84f8aed50c4b3e8e5b33c5b587b26e) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:38.289613 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f693a760-e01e-4083-9721-0dbc82b00001 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:38.291102 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f693a760-e01e-4083-9721-0dbc82b00001 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:38.297963 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 279/544] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:58:38 2022] GET /identity/v3/role_assignments?user.id=5a84f8aed50c4b3e8e5b33c5b587b26e&scope.project.id=b1177a85a0de4276898daf4a24ff069f&role.id=cfc67444e9944ffc87173be8c9245108 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:38.460465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c0bae-2d4e-47cf-b757-688686a76e8c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:38.516045 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9fda15-9ccc-4cac-a0b4-119b87b77d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:38.516749 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fda15-9ccc-4cac-a0b4-119b87b77d22 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:38.517034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fda15-9ccc-4cac-a0b4-119b87b77d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:38.517308 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fda15-9ccc-4cac-a0b4-119b87b77d22 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:38.518110 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9fda15-9ccc-4cac-a0b4-119b87b77d22 None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:38.518419 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9fda15-9ccc-4cac-a0b4-119b87b77d22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:38.519864 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9fda15-9ccc-4cac-a0b4-119b87b77d22 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:38.547538 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 266/545] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 15:58:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:38.553556 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693a760-e01e-4083-9721-0dbc82b00001 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:38.562736 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db77c2-105f-47ba-8975-8b4108d57a46 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:38.563101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db77c2-105f-47ba-8975-8b4108d57a46 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:38.563310 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db77c2-105f-47ba-8975-8b4108d57a46 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:38.564778 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 267/546] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:38.570615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ddf9e-995c-4942-ba41-720f21775fa5 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:38.570963 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ddf9e-995c-4942-ba41-720f21775fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:38.571186 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ddf9e-995c-4942-ba41-720f21775fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:38.602100 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b74ddf9e-995c-4942-ba41-720f21775fa5 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:38.603484 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b74ddf9e-995c-4942-ba41-720f21775fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:38.726304 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693a760-e01e-4083-9721-0dbc82b00001 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:38.730787 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:38.744867 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:38.756968 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 268/547] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:58:38.763887 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e8da1-4dd3-40f0-a01e-01936f54bb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:38.764684 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e8da1-4dd3-40f0-a01e-01936f54bb2a None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:38.765098 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e8da1-4dd3-40f0-a01e-01936f54bb2a None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:38.765385 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e8da1-4dd3-40f0-a01e-01936f54bb2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:38.781526 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ddf9e-995c-4942-ba41-720f21775fa5 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:38.785693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60e8da1-4dd3-40f0-a01e-01936f54bb2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:38.787287 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60e8da1-4dd3-40f0-a01e-01936f54bb2a None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:38.818467 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-a60e8da1-4dd3-40f0-a01e-01936f54bb2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:38.818467 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:38.818467 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:38.818467 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:38.818467 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:38.818467 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:38.818467 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:38.818467 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:38.818467 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:38.818467 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:38.818467 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:38.818467 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:38.818467 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:38.818467 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:38.818467 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:38.820539 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:38.820539 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:38.820539 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:38.820539 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:38.820539 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:38.820539 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:38.820539 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:38.820539 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:38.820539 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:38.820539 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:38.820539 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:38.820539 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:38.820539 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:38.820539 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:38.820539 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:38.820539 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:38.820539 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:38.822586 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:38.822586 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:38.822586 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:38.822586 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:38.826898 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 280/548] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:38.830397 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e8da1-4dd3-40f0-a01e-01936f54bb2a None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:38.972806 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e8da1-4dd3-40f0-a01e-01936f54bb2a None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:38.976083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ddf9e-995c-4942-ba41-720f21775fa5 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:39.024491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f380b6-fc95-4e34-b9f1-a16591bed9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:39.025079 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f380b6-fc95-4e34-b9f1-a16591bed9f3 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:39.025359 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f380b6-fc95-4e34-b9f1-a16591bed9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:39.025632 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f380b6-fc95-4e34-b9f1-a16591bed9f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:39.026956 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f380b6-fc95-4e34-b9f1-a16591bed9f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:39.027266 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f380b6-fc95-4e34-b9f1-a16591bed9f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:39.028661 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f380b6-fc95-4e34-b9f1-a16591bed9f3 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:39.044741 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 281/549] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:39.050510 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f380b6-fc95-4e34-b9f1-a16591bed9f3 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:39.079570 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab724c-92f1-490c-becd-4194b83c5c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:39.080390 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab724c-92f1-490c-becd-4194b83c5c35 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:39.080673 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab724c-92f1-490c-becd-4194b83c5c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:39.080951 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab724c-92f1-490c-becd-4194b83c5c35 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:39.085634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:39.122785 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab724c-92f1-490c-becd-4194b83c5c35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:39.132690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-edab724c-92f1-490c-becd-4194b83c5c35 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:39.132690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:39.132690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:39.132690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:39.132690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:39.132690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:39.132690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:39.132690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:39.132690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:39.132690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:39.132690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:39.132690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:39.132690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:39.132690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:39.132690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:39.132690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:39.135575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:39.135575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:39.135575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:39.135575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:39.135575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:39.135575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:39.135575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:39.135575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:39.135575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:39.135575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:39.135575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:39.135575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:39.135575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:39.135575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:39.135575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:39.135575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:39.137722 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:39.137722 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:39.137722 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:39.137722 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:39.137722 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:39.137722 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:39.137722 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:39.137722 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:39.137722 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:39.137722 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:39.137722 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:39.137722 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:39.137722 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 269/550] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 15:58:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:39.142507 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab724c-92f1-490c-becd-4194b83c5c35 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:39.203099 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f380b6-fc95-4e34-b9f1-a16591bed9f3 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:39.236946 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:39.269626 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03bba6-c660-4506-af80-76d1a65d109f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:39.270476 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03bba6-c660-4506-af80-76d1a65d109f None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:39.270766 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03bba6-c660-4506-af80-76d1a65d109f None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:39.271049 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03bba6-c660-4506-af80-76d1a65d109f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:39.288290 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d03bba6-c660-4506-af80-76d1a65d109f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:39.290104 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d03bba6-c660-4506-af80-76d1a65d109f None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:39.306403 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-7d03bba6-c660-4506-af80-76d1a65d109f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 15:58:39.306403 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:39.306403 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:39.306403 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:39.306403 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:39.306403 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:39.306403 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:39.306403 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:39.306403 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:39.306403 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:39.306403 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:39.306403 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:39.306403 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:58:39.306403 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:58:39.306403 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:58:39.306403 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:58:39.308390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:39.308390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:39.308390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:58:39.308390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:39.308390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:58:39.308390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:39.308390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:39.308390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:39.308390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:39.308390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:39.308390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:39.308390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:39.308390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:39.308390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:39.308390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:39.308390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:39.308390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:39.308390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:39.309745 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:39.309745 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:39.309745 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:58:39.309745 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:58:39.309745 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:58:39.309745 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:58:39.309745 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:58:39.309745 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:58:39.309745 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 15:58:39.309745 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:39.309745 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 282/551] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 15:58:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:39.314959 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03bba6-c660-4506-af80-76d1a65d109f None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:39.344830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab724c-92f1-490c-becd-4194b83c5c35 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:39.418878 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:39.422761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d8c76-5b1a-413e-89c8-07e6922caf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:39.423476 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d8c76-5b1a-413e-89c8-07e6922caf26 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:39.423769 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d8c76-5b1a-413e-89c8-07e6922caf26 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:39.424068 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d8c76-5b1a-413e-89c8-07e6922caf26 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:39.424823 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43d8c76-5b1a-413e-89c8-07e6922caf26 None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:39.425207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43d8c76-5b1a-413e-89c8-07e6922caf26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:39.426898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43d8c76-5b1a-413e-89c8-07e6922caf26 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:39.427549 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03bba6-c660-4506-af80-76d1a65d109f None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:39.450335 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 270/552] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 15:58:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:39.451876 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d8c76-5b1a-413e-89c8-07e6922caf26 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:39.468764 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe0bd1-569b-49b6-b788-cd220078cc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:39.469583 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe0bd1-569b-49b6-b788-cd220078cc3f None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:39.470056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe0bd1-569b-49b6-b788-cd220078cc3f None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:39.470441 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe0bd1-569b-49b6-b788-cd220078cc3f None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:39.471443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe0bd1-569b-49b6-b788-cd220078cc3f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:39.471868 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe0bd1-569b-49b6-b788-cd220078cc3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:39.473436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe0bd1-569b-49b6-b788-cd220078cc3f None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:39.492783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 283/553] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jun 16 15:58:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:39.494397 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5bbbe-f106-4dfe-8982-0523e6ea28f3 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:39.494791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5bbbe-f106-4dfe-8982-0523e6ea28f3 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:39.495173 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5bbbe-f106-4dfe-8982-0523e6ea28f3 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:39.496882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 284/554] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:39.497680 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5bbbe-f106-4dfe-8982-0523e6ea28f3 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:39.575320 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d8c76-5b1a-413e-89c8-07e6922caf26 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:39.589417 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:39.611662 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5bbbe-f106-4dfe-8982-0523e6ea28f3 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:39.615145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bdfa8d-be27-4ed1-a7fe-dd849d82fc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:39.615712 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bdfa8d-be27-4ed1-a7fe-dd849d82fc70 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:39.616009 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bdfa8d-be27-4ed1-a7fe-dd849d82fc70 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:39.616292 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bdfa8d-be27-4ed1-a7fe-dd849d82fc70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:39.627776 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bdfa8d-be27-4ed1-a7fe-dd849d82fc70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:39.629235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bdfa8d-be27-4ed1-a7fe-dd849d82fc70 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:39.647764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-37bdfa8d-be27-4ed1-a7fe-dd849d82fc70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:39.647764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:39.647764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:39.647764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:39.647764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:39.647764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:39.647764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:39.647764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:39.647764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:39.647764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:39.647764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:39.647764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:39.647764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:39.647764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:39.647764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:39.648677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:39.648677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:39.648677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:39.648677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:39.648677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:39.648677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:39.648677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:39.648677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:39.648677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:39.648677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:39.648677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:39.648677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:39.648677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:39.648677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:39.648677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:39.648677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:39.648677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:39.650194 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:39.650194 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:39.650194 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:39.650194 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:39.650194 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 271/555] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:39.650591 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e84de5-2302-443b-bf2f-79741f2a5bcf None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:39.650819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e84de5-2302-443b-bf2f-79741f2a5bcf None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:39.651095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e84de5-2302-443b-bf2f-79741f2a5bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:39.655677 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac692005-db76-4da6-bca5-1405d8b4daba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:39.656768 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac692005-db76-4da6-bca5-1405d8b4daba None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:39.657295 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac692005-db76-4da6-bca5-1405d8b4daba None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:39.657950 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac692005-db76-4da6-bca5-1405d8b4daba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:39.669068 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-14e84de5-2302-443b-bf2f-79741f2a5bcf None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:39.670444 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-14e84de5-2302-443b-bf2f-79741f2a5bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:39.687476 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:39.737747 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac692005-db76-4da6-bca5-1405d8b4daba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:39.739784 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac692005-db76-4da6-bca5-1405d8b4daba None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:39.747235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 272/556] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:39.748210 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e84de5-2302-443b-bf2f-79741f2a5bcf None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:39.867794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:39.875239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e84de5-2302-443b-bf2f-79741f2a5bcf None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:39.883701 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 285/557] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:39.884889 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac692005-db76-4da6-bca5-1405d8b4daba None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:39.917702 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c765ab-408a-4929-aca2-c5b0d64d04ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:39.918414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c765ab-408a-4929-aca2-c5b0d64d04ab None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:39.918845 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c765ab-408a-4929-aca2-c5b0d64d04ab None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:39.919245 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c765ab-408a-4929-aca2-c5b0d64d04ab None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:39.920847 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c765ab-408a-4929-aca2-c5b0d64d04ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:39.921314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c765ab-408a-4929-aca2-c5b0d64d04ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:39.922876 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c765ab-408a-4929-aca2-c5b0d64d04ab None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:39.942627 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 273/558] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:39.942627 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c765ab-408a-4929-aca2-c5b0d64d04ab None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:39.945625 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:40.008601 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac692005-db76-4da6-bca5-1405d8b4daba None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:40.047092 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ecb9a-6289-4110-bb6f-7e6472dc16df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:40.048254 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ecb9a-6289-4110-bb6f-7e6472dc16df None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:40.048758 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ecb9a-6289-4110-bb6f-7e6472dc16df None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:40.049238 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ecb9a-6289-4110-bb6f-7e6472dc16df None admin] PATH_INFO: `/v3/roles/service` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:40.060845 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ecb9a-6289-4110-bb6f-7e6472dc16df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:40.069239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-d44ecb9a-6289-4110-bb6f-7e6472dc16df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:40.069239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:40.069239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:40.069239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:40.069239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:40.069239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:40.069239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:40.069239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:40.069239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:40.069239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:40.069239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:40.069239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:40.069239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:40.069239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:40.069239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:40.069239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:40.070686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:40.070686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:40.070686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:40.070686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:40.070686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:40.070686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:40.070686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:40.070686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:40.070686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:40.070686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:40.070686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:40.070686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:40.070686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:40.070686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:40.070686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:40.070686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:40.071526 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:40.071526 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:40.071526 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:40.071526 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:40.071526 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:40.071526 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:40.071526 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:40.071526 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:40.071526 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:40.071526 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:40.071526 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:40.071526 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:40.074299 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 286/559] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 15:58:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:40.075408 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ecb9a-6289-4110-bb6f-7e6472dc16df None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:40.085997 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c765ab-408a-4929-aca2-c5b0d64d04ab None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:40.126021 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47224fb1-f977-4c90-a4d1-43a0dc6f40cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:40.127131 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47224fb1-f977-4c90-a4d1-43a0dc6f40cf None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:40.127594 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47224fb1-f977-4c90-a4d1-43a0dc6f40cf None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:40.128064 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47224fb1-f977-4c90-a4d1-43a0dc6f40cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:40.140655 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47224fb1-f977-4c90-a4d1-43a0dc6f40cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:40.143579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47224fb1-f977-4c90-a4d1-43a0dc6f40cf None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:40.162510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-47224fb1-f977-4c90-a4d1-43a0dc6f40cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:40.162510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:40.162510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:40.162510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:40.162510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:40.162510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:40.162510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:40.162510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:40.162510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:40.162510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:40.162510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:40.162510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:40.162510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:40.162510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:40.162510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:40.163445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:40.163445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:40.163445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:40.163445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:40.163445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:40.163445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:40.163445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:40.163445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:40.163445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:40.163445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:40.163445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:40.163445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:40.163445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:40.163445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:40.163445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:40.163445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:40.163445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:40.164435 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:40.164435 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:40.164435 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:40.164435 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:40.171862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 274/560] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:40.173083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47224fb1-f977-4c90-a4d1-43a0dc6f40cf None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:40.185206 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:40.214815 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ecb9a-6289-4110-bb6f-7e6472dc16df None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:40.225084 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:40.254052 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b87eb1-f9dd-4ab5-8106-7a1be2bffe29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:40.255225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b87eb1-f9dd-4ab5-8106-7a1be2bffe29 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:40.255723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b87eb1-f9dd-4ab5-8106-7a1be2bffe29 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:40.256219 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b87eb1-f9dd-4ab5-8106-7a1be2bffe29 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:40.266454 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b87eb1-f9dd-4ab5-8106-7a1be2bffe29 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:40.268301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b87eb1-f9dd-4ab5-8106-7a1be2bffe29 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:40.283962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-e2b87eb1-f9dd-4ab5-8106-7a1be2bffe29 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 15:58:40.283962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:40.283962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:40.283962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:40.283962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:40.283962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:40.283962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:40.283962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:40.283962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:40.283962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:40.283962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:40.283962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:40.283962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:58:40.283962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:58:40.283962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:58:40.283962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:58:40.284989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:40.284989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:40.284989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:58:40.284989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:40.284989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:58:40.284989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:40.284989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:40.284989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:40.284989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:40.284989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:40.284989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:40.284989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:40.284989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:40.284989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:40.284989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:40.284989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:40.284989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:40.284989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:40.285959 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:40.285959 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:40.285959 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:58:40.285959 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:58:40.285959 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:58:40.285959 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:58:40.285959 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:58:40.285959 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:58:40.285959 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 15:58:40.285959 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:40.293497 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 287/561] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jun 16 15:58:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:40.294827 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b87eb1-f9dd-4ab5-8106-7a1be2bffe29 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:40.303110 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47224fb1-f977-4c90-a4d1-43a0dc6f40cf None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:40.348924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1ef51-f7e1-4b97-a121-93561a231701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:40.350059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1ef51-f7e1-4b97-a121-93561a231701 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:40.350547 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1ef51-f7e1-4b97-a121-93561a231701 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:40.351048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1ef51-f7e1-4b97-a121-93561a231701 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:40.352117 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1ef51-f7e1-4b97-a121-93561a231701 None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:40.352805 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1ef51-f7e1-4b97-a121-93561a231701 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:40.354266 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1ef51-f7e1-4b97-a121-93561a231701 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:40.364059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 275/562] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 15:58:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:40.365088 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1ef51-f7e1-4b97-a121-93561a231701 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:40.418971 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b87eb1-f9dd-4ab5-8106-7a1be2bffe29 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:40.439098 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:40.455588 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fbb03-633e-4efb-9c6c-7355fda026f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:40.456260 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fbb03-633e-4efb-9c6c-7355fda026f4 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:40.456544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fbb03-633e-4efb-9c6c-7355fda026f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:40.456818 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fbb03-633e-4efb-9c6c-7355fda026f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:40.457548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1fbb03-633e-4efb-9c6c-7355fda026f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:40.457856 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1fbb03-633e-4efb-9c6c-7355fda026f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:40.459321 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1fbb03-633e-4efb-9c6c-7355fda026f4 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:40.472211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 288/563] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:40.473339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d555aa-3c97-46cb-a2cf-dcdf9906a274 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:40.473627 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d555aa-3c97-46cb-a2cf-dcdf9906a274 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:40.473900 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d555aa-3c97-46cb-a2cf-dcdf9906a274 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:40.478225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 289/564] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:40.478225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d555aa-3c97-46cb-a2cf-dcdf9906a274 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:40.480035 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1ef51-f7e1-4b97-a121-93561a231701 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:40.508313 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:40.520875 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6757b993-b27a-4e53-bcd2-05e63799823c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:40.521623 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757b993-b27a-4e53-bcd2-05e63799823c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:40.522119 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757b993-b27a-4e53-bcd2-05e63799823c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:40.522433 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757b993-b27a-4e53-bcd2-05e63799823c None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:40.523369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6757b993-b27a-4e53-bcd2-05e63799823c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:40.523690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6757b993-b27a-4e53-bcd2-05e63799823c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:40.525599 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6757b993-b27a-4e53-bcd2-05e63799823c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:40.546323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 276/565] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jun 16 15:58:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:40.547091 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6757b993-b27a-4e53-bcd2-05e63799823c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:40.595430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d555aa-3c97-46cb-a2cf-dcdf9906a274 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:40.635087 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b8615-3921-4546-8785-72c8b0aae441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:40.636318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b8615-3921-4546-8785-72c8b0aae441 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:40.636895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b8615-3921-4546-8785-72c8b0aae441 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:40.637243 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b8615-3921-4546-8785-72c8b0aae441 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:40.649776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b8615-3921-4546-8785-72c8b0aae441 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:40.651274 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b8615-3921-4546-8785-72c8b0aae441 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:40.655786 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6757b993-b27a-4e53-bcd2-05e63799823c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:40.671339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-249b8615-3921-4546-8785-72c8b0aae441 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:40.671339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:40.671339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:40.671339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:40.671339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:40.671339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:40.671339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:40.671339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:40.671339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:40.671339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:40.671339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:40.671339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:40.671339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:40.671339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:40.671339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:40.672443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:40.672443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:40.672443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:40.672443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:40.672443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:40.672443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:40.672443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:40.672443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:40.672443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:40.672443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:40.672443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:40.672443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:40.672443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:40.672443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:40.672443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:40.672443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:40.672443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:40.678264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:40.678264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:40.678264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:40.678264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:40.678264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 290/566] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:40.678264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c2aeb-8269-4f19-92a0-6fa53cc8507e None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:40.678264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c2aeb-8269-4f19-92a0-6fa53cc8507e None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:40.678264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c2aeb-8269-4f19-92a0-6fa53cc8507e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:40.692123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:40.695047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-623c2aeb-8269-4f19-92a0-6fa53cc8507e None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:40.696230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-623c2aeb-8269-4f19-92a0-6fa53cc8507e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:40.707879 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80515e-62b0-4f72-b9d7-5c6c7aa55c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:40.708698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80515e-62b0-4f72-b9d7-5c6c7aa55c42 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:40.709007 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80515e-62b0-4f72-b9d7-5c6c7aa55c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:40.709758 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80515e-62b0-4f72-b9d7-5c6c7aa55c42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:40.720491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80515e-62b0-4f72-b9d7-5c6c7aa55c42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:40.722709 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80515e-62b0-4f72-b9d7-5c6c7aa55c42 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:40.739882 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-1d80515e-62b0-4f72-b9d7-5c6c7aa55c42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:40.739882 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:40.739882 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:40.739882 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:40.739882 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:40.739882 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:40.739882 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:40.739882 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:40.739882 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:40.739882 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:40.739882 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:40.739882 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:40.739882 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:58:40.739882 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:58:40.739882 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:58:40.740841 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:58:40.740841 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:40.740841 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:40.740841 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:40.740841 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:40.740841 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:40.740841 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:40.740841 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:40.740841 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:40.740841 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:40.740841 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:40.740841 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:40.740841 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:40.740841 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:40.740841 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:40.740841 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:40.740841 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:40.741725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:58:40.741725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:58:40.741725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:58:40.741725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:58:40.741725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:58:40.741725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:58:40.741725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:40.741725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:40.741725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 277/567] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:40.742276 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80515e-62b0-4f72-b9d7-5c6c7aa55c42 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:40.811446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 291/568] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:40.812746 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c2aeb-8269-4f19-92a0-6fa53cc8507e None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:40.835916 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:40.887319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80515e-62b0-4f72-b9d7-5c6c7aa55c42 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:40.926426 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3e632-ec7b-4d5f-a875-ec31731eada6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:40.927600 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3e632-ec7b-4d5f-a875-ec31731eada6 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:40.928085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3e632-ec7b-4d5f-a875-ec31731eada6 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:40.928519 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3e632-ec7b-4d5f-a875-ec31731eada6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:40.940044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c2aeb-8269-4f19-92a0-6fa53cc8507e None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:40.940962 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3e632-ec7b-4d5f-a875-ec31731eada6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:40.942468 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3e632-ec7b-4d5f-a875-ec31731eada6 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:40.964257 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-87e3e632-ec7b-4d5f-a875-ec31731eada6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:40.964257 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:40.964257 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:40.964257 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:40.964257 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:40.964257 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:40.964257 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:40.964257 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:40.964257 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:40.964257 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:40.964257 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:40.964257 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:40.964257 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:40.964257 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:40.964257 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:40.965205 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:40.965205 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:40.965205 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:40.965205 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:40.965205 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:40.965205 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:40.965205 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:40.965205 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:40.965205 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:40.965205 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:40.965205 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:40.965205 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:40.965205 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:40.965205 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:40.965205 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:40.965205 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:40.965205 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:40.966129 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:40.966129 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:40.966129 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:40.966129 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:40.966129 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 278/569] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:40.966400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3e632-ec7b-4d5f-a875-ec31731eada6 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:40.982977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e0cc0-5388-4a25-9d01-f75dcc926dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:40.983653 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e0cc0-5388-4a25-9d01-f75dcc926dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:40.983979 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e0cc0-5388-4a25-9d01-f75dcc926dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:40.984186 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e0cc0-5388-4a25-9d01-f75dcc926dc6 None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:40.984912 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e0cc0-5388-4a25-9d01-f75dcc926dc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:40.985262 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e0cc0-5388-4a25-9d01-f75dcc926dc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:40.986723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e0cc0-5388-4a25-9d01-f75dcc926dc6 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:40.998842 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 292/570] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:41.004056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77772243-a855-4186-a567-a391cc8c6148 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:41.004356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77772243-a855-4186-a567-a391cc8c6148 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:41.004634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77772243-a855-4186-a567-a391cc8c6148 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:41.006128 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 293/571] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:41.006682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77772243-a855-4186-a567-a391cc8c6148 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:41.024980 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:41.083695 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:41.090781 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3e632-ec7b-4d5f-a875-ec31731eada6 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:41.118804 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77772243-a855-4186-a567-a391cc8c6148 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:41.124646 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8afdb72-cc43-452c-807e-2e0958ea3ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:41.125314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afdb72-cc43-452c-807e-2e0958ea3ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:41.125609 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afdb72-cc43-452c-807e-2e0958ea3ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:41.125887 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afdb72-cc43-452c-807e-2e0958ea3ce0 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:41.126702 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8afdb72-cc43-452c-807e-2e0958ea3ce0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:41.127014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8afdb72-cc43-452c-807e-2e0958ea3ce0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:41.128572 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8afdb72-cc43-452c-807e-2e0958ea3ce0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:41.141365 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 279/572] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:58:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:41.142220 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8afdb72-cc43-452c-807e-2e0958ea3ce0 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:41.160270 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f2604-bdd9-4e11-9faa-cd35b7f630c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:41.161067 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f2604-bdd9-4e11-9faa-cd35b7f630c4 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:41.161350 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f2604-bdd9-4e11-9faa-cd35b7f630c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:41.161628 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f2604-bdd9-4e11-9faa-cd35b7f630c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:41.171932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f2604-bdd9-4e11-9faa-cd35b7f630c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:41.179700 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-e61f2604-bdd9-4e11-9faa-cd35b7f630c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:41.179700 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:41.179700 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:41.179700 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:41.179700 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:41.179700 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:41.179700 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:41.179700 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:41.179700 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:41.179700 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:41.179700 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:41.179700 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:41.179700 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:41.179700 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:41.179700 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:41.179700 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:41.180645 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:41.180645 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:41.180645 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:41.180645 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:41.180645 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:41.180645 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:41.180645 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:41.180645 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:41.180645 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:41.180645 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:41.180645 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:41.180645 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:41.180645 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:41.180645 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:41.180645 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:41.180645 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:41.181484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:41.181484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:41.181484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:41.181484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:41.181484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:41.181484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:41.181484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:41.181484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:41.181484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:41.181484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:41.181484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:41.181484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:41.181484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 294/573] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 15:58:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:41.182202 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f2604-bdd9-4e11-9faa-cd35b7f630c4 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:41.251710 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:41.254911 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8afdb72-cc43-452c-807e-2e0958ea3ce0 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:41.288290 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f2604-bdd9-4e11-9faa-cd35b7f630c4 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:41.291542 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb0b09-0385-4e8c-890e-28fbf58c03bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:41.292212 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb0b09-0385-4e8c-890e-28fbf58c03bf None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:41.292501 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb0b09-0385-4e8c-890e-28fbf58c03bf None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:41.292776 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb0b09-0385-4e8c-890e-28fbf58c03bf None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:41.293503 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb0b09-0385-4e8c-890e-28fbf58c03bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:41.293813 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb0b09-0385-4e8c-890e-28fbf58c03bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:41.295471 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb0b09-0385-4e8c-890e-28fbf58c03bf None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:41.306855 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 280/574] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:41.308051 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c9aa6-7299-4d82-8035-1fa7e8d58c69 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:41.308344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c9aa6-7299-4d82-8035-1fa7e8d58c69 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:41.308625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c9aa6-7299-4d82-8035-1fa7e8d58c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:41.318962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:41.325260 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703099e-377a-4720-b51b-04c805cde30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:41.326069 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-7d4c9aa6-7299-4d82-8035-1fa7e8d58c69 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:41.326207 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703099e-377a-4720-b51b-04c805cde30b None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:41.327677 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703099e-377a-4720-b51b-04c805cde30b None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:41.327677 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703099e-377a-4720-b51b-04c805cde30b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:41.328008 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7d4c9aa6-7299-4d82-8035-1fa7e8d58c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:41.335423 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4703099e-377a-4720-b51b-04c805cde30b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:41.336887 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4703099e-377a-4720-b51b-04c805cde30b None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:41.351172 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-4703099e-377a-4720-b51b-04c805cde30b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 15:58:41.351172 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:41.351172 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:41.351172 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:41.351172 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:41.351172 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:41.351172 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:41.351172 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:41.351172 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:41.351172 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:41.351172 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:41.351172 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:41.351172 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:58:41.351172 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:58:41.351172 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:58:41.351172 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:58:41.352119 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:41.352119 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:41.352119 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:58:41.352119 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:41.352119 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:58:41.352119 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:41.352119 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:41.352119 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:41.352119 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:41.352119 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:41.352119 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:41.352119 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:41.352119 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:41.352119 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:41.352119 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:41.352119 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:41.352119 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:41.352119 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:41.353068 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:41.353068 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:41.353068 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:58:41.353068 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:58:41.353068 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:58:41.353068 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:58:41.353068 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:58:41.353068 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:58:41.353068 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 15:58:41.353068 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:41.353068 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 295/575] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:41.353659 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703099e-377a-4720-b51b-04c805cde30b None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:41.400890 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 281/576] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:41.401741 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c9aa6-7299-4d82-8035-1fa7e8d58c69 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:41.463845 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703099e-377a-4720-b51b-04c805cde30b None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:41.497934 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:41.501406 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85523275-6f7a-4c5a-97bf-5a8c792cc209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:41.502090 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85523275-6f7a-4c5a-97bf-5a8c792cc209 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:41.502383 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85523275-6f7a-4c5a-97bf-5a8c792cc209 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:41.502645 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85523275-6f7a-4c5a-97bf-5a8c792cc209 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:41.503463 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85523275-6f7a-4c5a-97bf-5a8c792cc209 None admin] RBAC: Adding query filter params (role.id=cfc67444e9944ffc87173be8c9245108, scope.project.id=b1177a85a0de4276898daf4a24ff069f, user.id=0082e37ce54f49f7837601349508c89f) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:41.503778 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85523275-6f7a-4c5a-97bf-5a8c792cc209 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:41.505215 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85523275-6f7a-4c5a-97bf-5a8c792cc209 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:41.510769 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c9aa6-7299-4d82-8035-1fa7e8d58c69 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:41.511086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 296/577] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:58:41 2022] GET /identity/v3/role_assignments?user.id=0082e37ce54f49f7837601349508c89f&scope.project.id=b1177a85a0de4276898daf4a24ff069f&role.id=cfc67444e9944ffc87173be8c9245108 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:41.511876 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85523275-6f7a-4c5a-97bf-5a8c792cc209 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:41.562858 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d026995-4637-443c-993b-655a463cb4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:41.563555 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d026995-4637-443c-993b-655a463cb4fe None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:41.563834 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d026995-4637-443c-993b-655a463cb4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:41.564107 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d026995-4637-443c-993b-655a463cb4fe None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:41.564873 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d026995-4637-443c-993b-655a463cb4fe None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:41.565176 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d026995-4637-443c-993b-655a463cb4fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:41.566808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d026995-4637-443c-993b-655a463cb4fe None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:41.579316 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 282/578] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 15:58:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:41.580483 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d026995-4637-443c-993b-655a463cb4fe None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:41.589557 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:41.645416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85523275-6f7a-4c5a-97bf-5a8c792cc209 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:41.689838 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fff4c6-59ff-43eb-9a96-4ea4abd54057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:41.690692 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fff4c6-59ff-43eb-9a96-4ea4abd54057 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:41.690976 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fff4c6-59ff-43eb-9a96-4ea4abd54057 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:41.691252 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fff4c6-59ff-43eb-9a96-4ea4abd54057 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:41.698857 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fff4c6-59ff-43eb-9a96-4ea4abd54057 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:41.700307 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fff4c6-59ff-43eb-9a96-4ea4abd54057 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:41.701018 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d026995-4637-443c-993b-655a463cb4fe None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:41.716723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-f3fff4c6-59ff-43eb-9a96-4ea4abd54057 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:41.716723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:41.716723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:41.716723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:41.716723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:41.716723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:41.716723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:41.716723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:41.716723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:41.716723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:41.716723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:41.716723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:41.716723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:58:41.716723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:58:41.716723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:58:41.718582 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:58:41.718582 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:41.718582 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:41.718582 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:41.718582 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:41.718582 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:41.718582 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:41.718582 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:41.718582 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:41.718582 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:41.718582 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:41.718582 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:41.718582 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:41.718582 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:41.718582 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:41.718582 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:41.718582 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:41.719488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:58:41.719488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:58:41.719488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:58:41.719488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:58:41.719488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:58:41.719488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:58:41.719488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:41.719488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:41.719913 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 297/579] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:41.720632 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fff4c6-59ff-43eb-9a96-4ea4abd54057 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:41.745995 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2b3fc-2145-47ce-b3b7-9be6117e2c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:41.746690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2b3fc-2145-47ce-b3b7-9be6117e2c63 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:41.746990 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2b3fc-2145-47ce-b3b7-9be6117e2c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:41.747249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2b3fc-2145-47ce-b3b7-9be6117e2c63 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:41.748340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2b3fc-2145-47ce-b3b7-9be6117e2c63 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:41.748493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2b3fc-2145-47ce-b3b7-9be6117e2c63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:41.750130 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2b3fc-2145-47ce-b3b7-9be6117e2c63 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:41.771310 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 283/580] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jun 16 15:58:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:41.772165 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2b3fc-2145-47ce-b3b7-9be6117e2c63 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:41.790419 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:41.845318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fff4c6-59ff-43eb-9a96-4ea4abd54057 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:41.860353 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:41.883804 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8de5b-58c0-4939-a7e0-12a7281a2844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:41.884601 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8de5b-58c0-4939-a7e0-12a7281a2844 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:41.884883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8de5b-58c0-4939-a7e0-12a7281a2844 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:41.885147 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8de5b-58c0-4939-a7e0-12a7281a2844 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:41.898810 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2b3fc-2145-47ce-b3b7-9be6117e2c63 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:41.899457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f8de5b-58c0-4939-a7e0-12a7281a2844 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:41.908013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-39f8de5b-58c0-4939-a7e0-12a7281a2844 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:41.908013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:41.908013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:41.908013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:41.908013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:41.908013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:41.908013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:41.908013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:41.908013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:41.908013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:41.908013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:41.908013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:41.908013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:41.908013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:41.908013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:41.908013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:41.908975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:41.908975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:41.908975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:41.908975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:41.908975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:41.908975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:41.908975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:41.908975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:41.908975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:41.908975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:41.908975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:41.908975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:41.908975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:41.908975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:41.908975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:41.908975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:41.909792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:41.909792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:41.909792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:41.909792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:41.909792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:41.909792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:41.909792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:41.909792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:41.909792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:41.909792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:41.909792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:41.909792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:41.909792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 298/581] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 15:58:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:41.911391 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8de5b-58c0-4939-a7e0-12a7281a2844 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:41.939538 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057bc07-8b16-48c8-adc2-ea052fe5acaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:41.940345 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057bc07-8b16-48c8-adc2-ea052fe5acaf None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:41.940635 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057bc07-8b16-48c8-adc2-ea052fe5acaf None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:41.940897 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057bc07-8b16-48c8-adc2-ea052fe5acaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:41.952130 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9057bc07-8b16-48c8-adc2-ea052fe5acaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:41.954192 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9057bc07-8b16-48c8-adc2-ea052fe5acaf None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:41.973101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-9057bc07-8b16-48c8-adc2-ea052fe5acaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:41.973101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:41.973101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:41.973101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:41.973101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:41.973101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:41.973101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:41.973101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:41.973101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:41.973101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:41.973101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:41.973101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:41.973101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:41.973101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:41.973101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:41.974226 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:41.974226 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:41.974226 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:41.974226 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:41.974226 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:41.974226 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:41.974226 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:41.974226 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:41.974226 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:41.974226 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:41.974226 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:41.974226 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:41.974226 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:41.974226 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:41.974226 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:41.974226 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:41.974226 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:41.975130 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:41.975130 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:41.975130 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:41.975130 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:41.975130 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 284/582] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:41.975130 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057bc07-8b16-48c8-adc2-ea052fe5acaf None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:42.036233 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8de5b-58c0-4939-a7e0-12a7281a2844 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:42.046888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:42.082342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ad009-facf-4d02-8ecc-8681958c2de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:42.083081 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ad009-facf-4d02-8ecc-8681958c2de7 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:42.083371 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ad009-facf-4d02-8ecc-8681958c2de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:42.083652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ad009-facf-4d02-8ecc-8681958c2de7 None admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:42.084477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ad009-facf-4d02-8ecc-8681958c2de7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:42.085012 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ad009-facf-4d02-8ecc-8681958c2de7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:42.086277 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ad009-facf-4d02-8ecc-8681958c2de7 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:42.090344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057bc07-8b16-48c8-adc2-ea052fe5acaf None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:42.104475 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 299/583] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:58:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:42.104475 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ad009-facf-4d02-8ecc-8681958c2de7 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:42.132726 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babca45e-056e-4286-8305-a0b595e68632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:42.133910 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babca45e-056e-4286-8305-a0b595e68632 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:42.134218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babca45e-056e-4286-8305-a0b595e68632 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:42.134474 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babca45e-056e-4286-8305-a0b595e68632 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:42.139916 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:42.148347 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babca45e-056e-4286-8305-a0b595e68632 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:42.149810 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babca45e-056e-4286-8305-a0b595e68632 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:42.168755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-babca45e-056e-4286-8305-a0b595e68632 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:42.168755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:42.168755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:42.168755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:42.168755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:42.168755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:42.168755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:42.168755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:42.168755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:42.168755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:42.168755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:42.168755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:42.168755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:42.168755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:42.168755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:42.169719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:42.169719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:42.169719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:42.169719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:42.169719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:42.169719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:42.169719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:42.169719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:42.169719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:42.169719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:42.169719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:42.169719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:42.169719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:42.169719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:42.169719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:42.169719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:42.169719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:42.171253 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:42.171253 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:42.171253 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:42.171253 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:42.175289 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 285/584] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:42.175998 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babca45e-056e-4286-8305-a0b595e68632 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:42.221881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ad009-facf-4d02-8ecc-8681958c2de7 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:42.261771 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e7ab6b-b3a0-4946-94ef-73b04cbb2e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:42.262399 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7ab6b-b3a0-4946-94ef-73b04cbb2e54 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:42.262683 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7ab6b-b3a0-4946-94ef-73b04cbb2e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:42.262956 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7ab6b-b3a0-4946-94ef-73b04cbb2e54 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:42.263754 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e7ab6b-b3a0-4946-94ef-73b04cbb2e54 None admin] RBAC: Adding query filter params (name=service) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:42.264709 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e7ab6b-b3a0-4946-94ef-73b04cbb2e54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:42.266117 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e7ab6b-b3a0-4946-94ef-73b04cbb2e54 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:42.280055 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 300/585] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 15:58:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:42.280055 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e7ab6b-b3a0-4946-94ef-73b04cbb2e54 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:42.289604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babca45e-056e-4286-8305-a0b595e68632 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:42.306289 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:42.329344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88b4c9-683e-4aca-ba4c-77d6844e607e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:42.330066 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88b4c9-683e-4aca-ba4c-77d6844e607e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:42.330356 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88b4c9-683e-4aca-ba4c-77d6844e607e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:42.330650 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88b4c9-683e-4aca-ba4c-77d6844e607e None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:42.331366 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c88b4c9-683e-4aca-ba4c-77d6844e607e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:42.331675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c88b4c9-683e-4aca-ba4c-77d6844e607e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:42.333059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c88b4c9-683e-4aca-ba4c-77d6844e607e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:42.346914 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 286/586] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:42.347737 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88b4c9-683e-4aca-ba4c-77d6844e607e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:42.388284 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:42.395861 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e7ab6b-b3a0-4946-94ef-73b04cbb2e54 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:42.445289 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f60c0-36e4-420f-9142-8ebc6947a0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:42.446342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f60c0-36e4-420f-9142-8ebc6947a0e6 None admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:42.446628 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f60c0-36e4-420f-9142-8ebc6947a0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:42.446909 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f60c0-36e4-420f-9142-8ebc6947a0e6 None admin] PATH_INFO: `/v3/projects/b1177a85a0de4276898daf4a24ff069f/users/c3645b1a0be34c01ac87c734fcd85ff6/roles/cfc67444e9944ffc87173be8c9245108` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:42.458309 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88b4c9-683e-4aca-ba4c-77d6844e607e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:42.464505 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f60c0-36e4-420f-9142-8ebc6947a0e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1177a85a0de4276898daf4a24ff069f, user_id=c3645b1a0be34c01ac87c734fcd85ff6, role_id=cfc67444e9944ffc87173be8c9245108)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:42.465992 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f60c0-36e4-420f-9142-8ebc6947a0e6 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:42.480502 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 301/587] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 15:58:42 2022] PUT /identity/v3/projects/b1177a85a0de4276898daf4a24ff069f/users/c3645b1a0be34c01ac87c734fcd85ff6/roles/cfc67444e9944ffc87173be8c9245108 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:58:42.490223 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f60c0-36e4-420f-9142-8ebc6947a0e6 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:42.496908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8067a-efa3-49c9-9296-75214c414e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:42.497599 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8067a-efa3-49c9-9296-75214c414e80 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:42.497946 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8067a-efa3-49c9-9296-75214c414e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:42.498440 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8067a-efa3-49c9-9296-75214c414e80 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:42.499949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8067a-efa3-49c9-9296-75214c414e80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:42.500321 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8067a-efa3-49c9-9296-75214c414e80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:42.501776 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8067a-efa3-49c9-9296-75214c414e80 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:42.520439 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 287/588] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:42.521469 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8067a-efa3-49c9-9296-75214c414e80 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:42.568579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:42.650362 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f60c0-36e4-420f-9142-8ebc6947a0e6 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:42.653761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8067a-efa3-49c9-9296-75214c414e80 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:42.698975 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c435700-11c5-40cd-83e3-a70baaa18b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:42.700366 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c435700-11c5-40cd-83e3-a70baaa18b56 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:42.700845 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c435700-11c5-40cd-83e3-a70baaa18b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:42.700974 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e7c35-8974-413e-a76c-d935f1b2539a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:42.701163 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c435700-11c5-40cd-83e3-a70baaa18b56 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:42.701802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e7c35-8974-413e-a76c-d935f1b2539a None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:42.702154 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e7c35-8974-413e-a76c-d935f1b2539a None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:42.702436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e7c35-8974-413e-a76c-d935f1b2539a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:42.711585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c435700-11c5-40cd-83e3-a70baaa18b56 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:42.713163 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c435700-11c5-40cd-83e3-a70baaa18b56 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:42.714602 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e7c35-8974-413e-a76c-d935f1b2539a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:42.715896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e7c35-8974-413e-a76c-d935f1b2539a None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:42.725740 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:42.736256 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-4c435700-11c5-40cd-83e3-a70baaa18b56 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 15:58:42.736256 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:42.736256 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:42.736256 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:42.736256 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:42.736256 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:42.736256 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:42.736256 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:42.736256 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:42.736256 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:42.736256 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:42.736256 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:42.736256 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:58:42.736256 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:58:42.736256 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:58:42.736256 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:58:42.737239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:42.737239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:42.737239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:58:42.737239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:42.737239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:58:42.737239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:42.737239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:42.737239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:42.737239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:42.737239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:42.737239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:42.737239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:42.737239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:42.737239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:42.737239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:42.737239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:42.737239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:42.737239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:42.738211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:42.738211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:42.738211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:58:42.738211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:58:42.738211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:58:42.738211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:58:42.738211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:58:42.738211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:58:42.738211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 15:58:42.738211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:42.738211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 288/589] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:42.738816 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c435700-11c5-40cd-83e3-a70baaa18b56 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:42.752395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-c96e7c35-8974-413e-a76c-d935f1b2539a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:42.752395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:42.752395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:42.752395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:42.752395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:42.752395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:42.752395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:42.752395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:42.752395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:42.752395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:42.752395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:42.752395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:42.752395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:42.752395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:42.752395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:42.753317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:42.753317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:42.753317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:42.753317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:42.753317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:42.753317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:42.753317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:42.753317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:42.753317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:42.753317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:42.753317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:42.753317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:42.753317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:42.753317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:42.753317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:42.753317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:42.753317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:42.754315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:42.754315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:42.754315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:42.754315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:42.756568 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 302/590] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:42.757670 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e7c35-8974-413e-a76c-d935f1b2539a None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:42.857963 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:42.867653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c435700-11c5-40cd-83e3-a70baaa18b56 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:42.880081 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e7c35-8974-413e-a76c-d935f1b2539a None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:42.914665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6989aa-4187-47dc-83eb-960dbbd1ad32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:42.915754 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6989aa-4187-47dc-83eb-960dbbd1ad32 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:42.916176 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6989aa-4187-47dc-83eb-960dbbd1ad32 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:42.916589 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6989aa-4187-47dc-83eb-960dbbd1ad32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:42.918463 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19840ee9-0123-4502-9478-e34d4dab8628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:42.919197 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19840ee9-0123-4502-9478-e34d4dab8628 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:42.919485 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19840ee9-0123-4502-9478-e34d4dab8628 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:42.919822 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19840ee9-0123-4502-9478-e34d4dab8628 None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:42.920750 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19840ee9-0123-4502-9478-e34d4dab8628 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:42.921063 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19840ee9-0123-4502-9478-e34d4dab8628 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:42.922579 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19840ee9-0123-4502-9478-e34d4dab8628 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:42.926524 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6989aa-4187-47dc-83eb-960dbbd1ad32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:42.928132 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6989aa-4187-47dc-83eb-960dbbd1ad32 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:42.943264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 303/591] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jun 16 15:58:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:42.944365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19840ee9-0123-4502-9478-e34d4dab8628 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:42.945865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-4c6989aa-4187-47dc-83eb-960dbbd1ad32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:42.945865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:42.945865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:42.945865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:42.945865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:42.945865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:42.945865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:42.945865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:42.945865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:42.945865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:42.945865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:42.945865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:42.945865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:58:42.945865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:58:42.945865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:58:42.946794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:58:42.946794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:42.946794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:42.946794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:42.946794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:42.946794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:42.946794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:42.946794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:42.946794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:42.946794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:42.946794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:42.946794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:42.946794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:42.946794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:42.946794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:42.946794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:42.946794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:42.947688 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:58:42.947688 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:58:42.947688 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:58:42.947688 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:58:42.947688 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:58:42.947688 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:58:42.947688 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:42.947688 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:42.950616 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 289/592] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:42.954171 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6989aa-4187-47dc-83eb-960dbbd1ad32 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:43.026015 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:43.070228 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6989aa-4187-47dc-83eb-960dbbd1ad32 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:43.104088 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19840ee9-0123-4502-9478-e34d4dab8628 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:43.118830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2623f646-5524-4ebc-883e-0728afce577b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:43.120252 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623f646-5524-4ebc-883e-0728afce577b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:43.121190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623f646-5524-4ebc-883e-0728afce577b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:43.121782 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623f646-5524-4ebc-883e-0728afce577b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:43.139935 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2623f646-5524-4ebc-883e-0728afce577b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:43.142231 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2623f646-5524-4ebc-883e-0728afce577b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:43.144766 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c481e80-9275-42cf-9209-e22fa92b1c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:43.145571 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c481e80-9275-42cf-9209-e22fa92b1c3d None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:43.145880 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c481e80-9275-42cf-9209-e22fa92b1c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:43.146240 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c481e80-9275-42cf-9209-e22fa92b1c3d None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:43.147197 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c481e80-9275-42cf-9209-e22fa92b1c3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:43.147576 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c481e80-9275-42cf-9209-e22fa92b1c3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:43.149025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c481e80-9275-42cf-9209-e22fa92b1c3d None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:43.161594 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 304/593] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:43.162678 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c481e80-9275-42cf-9209-e22fa92b1c3d None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:43.164789 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-2623f646-5524-4ebc-883e-0728afce577b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:43.164789 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:43.164789 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:43.164789 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:43.164789 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:43.164789 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:43.164789 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:43.164789 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:43.164789 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:43.164789 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:43.164789 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:43.164789 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:43.164789 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:43.164789 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:43.164789 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:43.165658 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:43.165658 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:43.165658 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:43.165658 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:43.165658 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:43.165658 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:43.165658 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:43.165658 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:43.165658 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:43.165658 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:43.165658 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:43.165658 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:43.165658 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:43.165658 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:43.165658 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:43.165658 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:43.165658 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:43.166582 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:43.166582 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:43.166582 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:43.166582 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:43.169184 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 290/594] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:43.169303 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2623f646-5524-4ebc-883e-0728afce577b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:43.184645 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:43.288122 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c481e80-9275-42cf-9209-e22fa92b1c3d None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:43.304643 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2623f646-5524-4ebc-883e-0728afce577b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:43.313410 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:43.339926 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb8ca7-93fd-4330-98f1-c420c170ea1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:43.341292 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb8ca7-93fd-4330-98f1-c420c170ea1f None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:43.341781 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb8ca7-93fd-4330-98f1-c420c170ea1f None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:43.342477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb8ca7-93fd-4330-98f1-c420c170ea1f None admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:43.343682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb8ca7-93fd-4330-98f1-c420c170ea1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:43.344197 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb8ca7-93fd-4330-98f1-c420c170ea1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:43.344646 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd33faf-9cc3-4fef-9d0a-922951703973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:43.345459 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd33faf-9cc3-4fef-9d0a-922951703973 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:43.345743 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd33faf-9cc3-4fef-9d0a-922951703973 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:43.346067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd33faf-9cc3-4fef-9d0a-922951703973 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:43.347071 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb8ca7-93fd-4330-98f1-c420c170ea1f None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:43.354549 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd33faf-9cc3-4fef-9d0a-922951703973 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:43.356004 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd33faf-9cc3-4fef-9d0a-922951703973 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:43.369019 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 305/595] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:58:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:43.370270 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb8ca7-93fd-4330-98f1-c420c170ea1f None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:43.372451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-3fd33faf-9cc3-4fef-9d0a-922951703973 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 15:58:43.372451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:43.372451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:43.372451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:43.372451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:43.372451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:43.372451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:43.372451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:43.372451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:43.372451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:43.372451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:43.372451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:43.372451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:58:43.372451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:58:43.372451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:58:43.372451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:58:43.374246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:43.374246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:43.374246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:58:43.374246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:43.374246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:58:43.374246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:43.374246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:43.374246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:43.374246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:43.374246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:43.374246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:43.374246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:43.374246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:43.374246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:43.374246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:43.374246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:43.374246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:43.374246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:43.375197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:43.375197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:43.375197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:58:43.375197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:58:43.375197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:58:43.375197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:58:43.375197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:58:43.375197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:58:43.375197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 15:58:43.375197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:43.375197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 291/596] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 15:58:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:43.375819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd33faf-9cc3-4fef-9d0a-922951703973 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:43.469144 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:43.509113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd33faf-9cc3-4fef-9d0a-922951703973 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:43.510554 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb8ca7-93fd-4330-98f1-c420c170ea1f None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:43.546368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacdefc0-288e-48ab-8ed5-1691a9fde17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:43.547412 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacdefc0-288e-48ab-8ed5-1691a9fde17e None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:43.547768 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacdefc0-288e-48ab-8ed5-1691a9fde17e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:43.548118 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacdefc0-288e-48ab-8ed5-1691a9fde17e None admin] PATH_INFO: `/v3/projects/b1177a85a0de4276898daf4a24ff069f/users/6f11beca9c5148eea4cc45f8dc89dbda/roles/cfc67444e9944ffc87173be8c9245108` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:43.558507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e768c10-67c7-4e7b-a206-f5f54fc2e9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:43.559177 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e768c10-67c7-4e7b-a206-f5f54fc2e9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:43.559459 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e768c10-67c7-4e7b-a206-f5f54fc2e9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:43.559732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e768c10-67c7-4e7b-a206-f5f54fc2e9d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:43.560462 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e768c10-67c7-4e7b-a206-f5f54fc2e9d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:43.561417 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e768c10-67c7-4e7b-a206-f5f54fc2e9d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:43.562862 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e768c10-67c7-4e7b-a206-f5f54fc2e9d3 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:43.569519 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacdefc0-288e-48ab-8ed5-1691a9fde17e None admin] RBAC: Authorizing `identity:create_grant(project_id=b1177a85a0de4276898daf4a24ff069f, user_id=6f11beca9c5148eea4cc45f8dc89dbda, role_id=cfc67444e9944ffc87173be8c9245108)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:43.571832 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacdefc0-288e-48ab-8ed5-1691a9fde17e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:43.575899 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 306/597] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:43.576934 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e768c10-67c7-4e7b-a206-f5f54fc2e9d3 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:43.592390 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 292/598] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 15:58:43 2022] PUT /identity/v3/projects/b1177a85a0de4276898daf4a24ff069f/users/6f11beca9c5148eea4cc45f8dc89dbda/roles/cfc67444e9944ffc87173be8c9245108 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:58:43.593342 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacdefc0-288e-48ab-8ed5-1691a9fde17e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:43.666597 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:43.775345 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e768c10-67c7-4e7b-a206-f5f54fc2e9d3 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:43.792337 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacdefc0-288e-48ab-8ed5-1691a9fde17e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:43.815366 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:43.838951 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffeaf37-5e5b-40b0-9424-f2251ebf2e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:43.839928 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffeaf37-5e5b-40b0-9424-f2251ebf2e60 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:43.840209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffeaf37-5e5b-40b0-9424-f2251ebf2e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:43.840482 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffeaf37-5e5b-40b0-9424-f2251ebf2e60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:43.840585 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c45ada-e9cf-446f-9033-11843796a44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:43.841249 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c45ada-e9cf-446f-9033-11843796a44e None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:43.841535 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c45ada-e9cf-446f-9033-11843796a44e None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:43.841834 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c45ada-e9cf-446f-9033-11843796a44e None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:43.843824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c45ada-e9cf-446f-9033-11843796a44e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:43.844087 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c45ada-e9cf-446f-9033-11843796a44e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:43.845611 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c45ada-e9cf-446f-9033-11843796a44e None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:43.847924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffeaf37-5e5b-40b0-9424-f2251ebf2e60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:43.849366 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffeaf37-5e5b-40b0-9424-f2251ebf2e60 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:43.863661 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-8ffeaf37-5e5b-40b0-9424-f2251ebf2e60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:43.863661 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:43.863661 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:43.863661 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:43.863661 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:43.863661 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:43.863661 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:43.863661 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:43.863661 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:43.863661 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:43.863661 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:43.863661 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:43.863661 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:58:43.863661 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:58:43.863661 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:58:43.864579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:58:43.864579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:43.864579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:43.864579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:43.864579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:43.864579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:43.864579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:43.864579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:43.864579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:43.864579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:43.864579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:43.864579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:43.864579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:43.864579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:43.864579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:43.864579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:43.864579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:43.865452 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:58:43.865452 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:58:43.865452 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:58:43.865452 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:58:43.865452 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:58:43.865452 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:58:43.865452 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:43.865452 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:43.865452 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 293/599] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:43.865974 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 307/600] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jun 16 15:58:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:43.869977 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2233515-35a3-4c54-ae4a-491d97fd0483 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:43.869977 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2233515-35a3-4c54-ae4a-491d97fd0483 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:43.869977 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2233515-35a3-4c54-ae4a-491d97fd0483 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:43.869977 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 294/601] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:43.874953 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2233515-35a3-4c54-ae4a-491d97fd0483 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:43.875446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ceb110-86a9-4298-b94d-429e832e21cb None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:43.875743 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ceb110-86a9-4298-b94d-429e832e21cb None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:43.876026 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ceb110-86a9-4298-b94d-429e832e21cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:43.898197 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-35ceb110-86a9-4298-b94d-429e832e21cb None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:43.899591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-35ceb110-86a9-4298-b94d-429e832e21cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:43.993012 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2233515-35a3-4c54-ae4a-491d97fd0483 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:43.994467 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 308/602] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:44.002407 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ceb110-86a9-4298-b94d-429e832e21cb None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:44.040447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f523a-27e6-4543-8409-bccb50d1bd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:44.041246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f523a-27e6-4543-8409-bccb50d1bd8a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:44.041524 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f523a-27e6-4543-8409-bccb50d1bd8a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:44.041796 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f523a-27e6-4543-8409-bccb50d1bd8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:44.045490 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:44.057459 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f523a-27e6-4543-8409-bccb50d1bd8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:44.058936 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f523a-27e6-4543-8409-bccb50d1bd8a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:44.077950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-f03f523a-27e6-4543-8409-bccb50d1bd8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:44.077950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:44.077950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:44.077950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:44.077950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:44.077950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:44.077950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:44.077950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:44.077950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:44.077950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:44.077950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:44.077950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:44.077950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:44.077950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:44.077950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:44.078908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:44.078908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:44.078908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:44.078908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:44.078908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:44.078908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:44.078908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:44.078908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:44.078908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:44.078908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:44.078908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:44.078908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:44.078908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:44.078908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:44.078908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:44.078908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:44.078908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:44.079789 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:44.079789 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:44.079789 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:44.079789 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:44.079789 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 295/603] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:44.080493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f523a-27e6-4543-8409-bccb50d1bd8a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:44.111312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:44.143006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ceb110-86a9-4298-b94d-429e832e21cb None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:44.184019 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29b509-e265-473e-bfe1-d8034e0ea255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:44.185247 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29b509-e265-473e-bfe1-d8034e0ea255 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:44.186484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29b509-e265-473e-bfe1-d8034e0ea255 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:44.187098 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29b509-e265-473e-bfe1-d8034e0ea255 None admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:44.188110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29b509-e265-473e-bfe1-d8034e0ea255 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:44.188624 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29b509-e265-473e-bfe1-d8034e0ea255 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:44.188729 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f523a-27e6-4543-8409-bccb50d1bd8a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:44.190593 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29b509-e265-473e-bfe1-d8034e0ea255 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:44.204140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 309/604] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:58:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:44.205504 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29b509-e265-473e-bfe1-d8034e0ea255 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:44.224067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c2051-245d-4abd-bd0c-46b2c895956c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:44.224762 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c2051-245d-4abd-bd0c-46b2c895956c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:44.225042 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c2051-245d-4abd-bd0c-46b2c895956c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:44.225310 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c2051-245d-4abd-bd0c-46b2c895956c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:44.236605 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c2051-245d-4abd-bd0c-46b2c895956c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:44.244274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-cd2c2051-245d-4abd-bd0c-46b2c895956c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:44.244274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:44.244274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:44.244274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:44.244274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:44.244274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:44.244274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:44.244274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:44.244274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:44.244274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:44.244274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:44.244274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:44.244274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:44.244274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:44.244274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:44.244274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:44.245222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:44.245222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:44.245222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:44.245222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:44.245222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:44.245222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:44.245222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:44.245222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:44.245222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:44.245222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:44.245222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:44.245222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:44.245222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:44.245222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:44.245222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:44.245222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:44.246096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:44.246096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:44.246096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:44.246096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:44.246096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:44.246096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:44.246096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:44.246096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:44.246096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:44.246096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:44.246096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:44.246096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:44.246096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 296/605] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 15:58:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:44.246807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c2051-245d-4abd-bd0c-46b2c895956c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:44.313323 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:44.332495 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29b509-e265-473e-bfe1-d8034e0ea255 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:44.341711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:44.355168 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c2051-245d-4abd-bd0c-46b2c895956c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:44.374529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f0d85-9d95-44bc-bbd2-d9172bbfa6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:44.375541 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f0d85-9d95-44bc-bbd2-d9172bbfa6c1 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:44.376136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f0d85-9d95-44bc-bbd2-d9172bbfa6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:44.376611 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f0d85-9d95-44bc-bbd2-d9172bbfa6c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:44.377545 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f0d85-9d95-44bc-bbd2-d9172bbfa6c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:44.378085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f0d85-9d95-44bc-bbd2-d9172bbfa6c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:44.379862 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f0d85-9d95-44bc-bbd2-d9172bbfa6c1 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:44.391884 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d4464-0fcc-421a-a9c6-feae3fb0d9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:44.392581 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d4464-0fcc-421a-a9c6-feae3fb0d9a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:44.392860 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d4464-0fcc-421a-a9c6-feae3fb0d9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:44.393131 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d4464-0fcc-421a-a9c6-feae3fb0d9a4 None admin] PATH_INFO: `/v3/projects/b1177a85a0de4276898daf4a24ff069f/users/69b669a5ba4e4294baeea369949e3e6f/roles/cfc67444e9944ffc87173be8c9245108` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:44.396590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 310/606] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:44.397671 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f0d85-9d95-44bc-bbd2-d9172bbfa6c1 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:44.409295 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06d4464-0fcc-421a-a9c6-feae3fb0d9a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1177a85a0de4276898daf4a24ff069f, user_id=69b669a5ba4e4294baeea369949e3e6f, role_id=cfc67444e9944ffc87173be8c9245108)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:44.410801 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06d4464-0fcc-421a-a9c6-feae3fb0d9a4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:44.425939 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 297/607] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 15:58:44 2022] PUT /identity/v3/projects/b1177a85a0de4276898daf4a24ff069f/users/69b669a5ba4e4294baeea369949e3e6f/roles/cfc67444e9944ffc87173be8c9245108 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:58:44.426924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d4464-0fcc-421a-a9c6-feae3fb0d9a4 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:44.521616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f0d85-9d95-44bc-bbd2-d9172bbfa6c1 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:44.545731 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d4464-0fcc-421a-a9c6-feae3fb0d9a4 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:44.563380 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4f666-ae74-48a9-a81a-b8f4744d8a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:44.564614 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4f666-ae74-48a9-a81a-b8f4744d8a1f None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:44.565166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4f666-ae74-48a9-a81a-b8f4744d8a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:44.565642 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4f666-ae74-48a9-a81a-b8f4744d8a1f None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:44.575026 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d4f666-ae74-48a9-a81a-b8f4744d8a1f None admin] RBAC: Adding query filter params (name=service) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:44.575683 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d4f666-ae74-48a9-a81a-b8f4744d8a1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:44.577919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d4f666-ae74-48a9-a81a-b8f4744d8a1f None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:44.582707 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a83971-1e1e-40b1-ba06-96161fa9b01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:44.583506 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a83971-1e1e-40b1-ba06-96161fa9b01b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:44.583783 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a83971-1e1e-40b1-ba06-96161fa9b01b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:44.584054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a83971-1e1e-40b1-ba06-96161fa9b01b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:44.591463 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a83971-1e1e-40b1-ba06-96161fa9b01b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:44.593814 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a83971-1e1e-40b1-ba06-96161fa9b01b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:44.596384 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 311/608] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 15:58:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:44.602356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4f666-ae74-48a9-a81a-b8f4744d8a1f None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:44.602961 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:44.615521 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:44.622067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-59a83971-1e1e-40b1-ba06-96161fa9b01b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:44.622067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:44.622067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:44.622067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:44.622067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:44.622067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:44.622067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:44.622067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:44.622067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:44.622067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:44.622067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:44.622067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:44.622067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:58:44.622067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:58:44.622067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:58:44.622955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:58:44.622955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:44.622955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:44.622955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:44.622955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:44.622955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:44.622955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:44.622955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:44.622955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:44.622955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:44.622955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:44.622955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:44.622955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:44.622955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:44.622955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:44.622955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:44.622955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:44.623950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:58:44.623950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:58:44.623950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:58:44.623950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:58:44.623950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:58:44.623950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:58:44.623950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:44.623950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:44.623950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 298/609] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:44.632559 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a83971-1e1e-40b1-ba06-96161fa9b01b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:44.747202 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4f666-ae74-48a9-a81a-b8f4744d8a1f None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:44.787675 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14363d-ba06-4b90-ab4f-064c2f1201d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:44.788941 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14363d-ba06-4b90-ab4f-064c2f1201d8 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:44.789433 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14363d-ba06-4b90-ab4f-064c2f1201d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:44.789957 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14363d-ba06-4b90-ab4f-064c2f1201d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:44.805962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14363d-ba06-4b90-ab4f-064c2f1201d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:44.805962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14363d-ba06-4b90-ab4f-064c2f1201d8 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:44.822589 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a83971-1e1e-40b1-ba06-96161fa9b01b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:44.824980 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-1d14363d-ba06-4b90-ab4f-064c2f1201d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:44.824980 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:44.824980 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:44.824980 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:44.824980 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:44.824980 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:44.824980 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:44.824980 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:44.824980 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:44.824980 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:44.824980 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:44.824980 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:44.824980 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:44.824980 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:44.824980 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:44.826682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:44.826682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:44.826682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:44.826682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:44.826682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:44.826682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:44.826682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:44.826682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:44.826682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:44.826682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:44.826682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:44.826682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:44.826682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:44.826682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:44.826682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:44.826682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:44.826682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:44.828286 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:44.828286 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:44.828286 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:44.828286 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:44.828286 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 312/610] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:58:44.832214 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14363d-ba06-4b90-ab4f-064c2f1201d8 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:44.866155 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09efd6e3-428f-4036-b67b-462a76c2af21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:44.866928 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efd6e3-428f-4036-b67b-462a76c2af21 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:44.867293 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efd6e3-428f-4036-b67b-462a76c2af21 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:44.867532 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efd6e3-428f-4036-b67b-462a76c2af21 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:44.868403 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09efd6e3-428f-4036-b67b-462a76c2af21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:44.868712 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09efd6e3-428f-4036-b67b-462a76c2af21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:44.870345 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09efd6e3-428f-4036-b67b-462a76c2af21 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:44.884970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 299/611] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:58:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:44.893610 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09efd6e3-428f-4036-b67b-462a76c2af21 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:44.901021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:44.947835 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14363d-ba06-4b90-ab4f-064c2f1201d8 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:44.986724 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458caee-e57b-43eb-8c5a-f7a1f79c4f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:44.987601 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458caee-e57b-43eb-8c5a-f7a1f79c4f8f None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:44.987954 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458caee-e57b-43eb-8c5a-f7a1f79c4f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:44.988297 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458caee-e57b-43eb-8c5a-f7a1f79c4f8f None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:44.989091 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e458caee-e57b-43eb-8c5a-f7a1f79c4f8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:44.989466 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e458caee-e57b-43eb-8c5a-f7a1f79c4f8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:44.990955 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e458caee-e57b-43eb-8c5a-f7a1f79c4f8f None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:45.004244 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 313/612] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:45.009857 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458caee-e57b-43eb-8c5a-f7a1f79c4f8f None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:45.055284 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:45.118074 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09efd6e3-428f-4036-b67b-462a76c2af21 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:45.164948 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458caee-e57b-43eb-8c5a-f7a1f79c4f8f None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:45.178692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a12792-81af-4e88-ab86-b91725b49279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:45.180684 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a12792-81af-4e88-ab86-b91725b49279 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:45.181503 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a12792-81af-4e88-ab86-b91725b49279 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:45.182247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a12792-81af-4e88-ab86-b91725b49279 None admin] PATH_INFO: `/v3/projects/b1177a85a0de4276898daf4a24ff069f/users/5a84f8aed50c4b3e8e5b33c5b587b26e/roles/cfc67444e9944ffc87173be8c9245108` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:45.184888 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:45.205369 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ff0445-2b43-483b-9120-644ff839b913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:45.206387 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff0445-2b43-483b-9120-644ff839b913 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:45.206766 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff0445-2b43-483b-9120-644ff839b913 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:45.207138 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff0445-2b43-483b-9120-644ff839b913 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:45.217237 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ff0445-2b43-483b-9120-644ff839b913 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:45.218953 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a12792-81af-4e88-ab86-b91725b49279 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1177a85a0de4276898daf4a24ff069f, user_id=5a84f8aed50c4b3e8e5b33c5b587b26e, role_id=cfc67444e9944ffc87173be8c9245108)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:45.219473 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ff0445-2b43-483b-9120-644ff839b913 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:45.222332 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a12792-81af-4e88-ab86-b91725b49279 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:45.238749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-f0ff0445-2b43-483b-9120-644ff839b913 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 15:58:45.238749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:45.238749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:45.238749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:45.238749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:45.238749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:45.238749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:45.238749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:45.238749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:45.238749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:45.238749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:45.238749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:45.238749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:58:45.238749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:58:45.238749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:58:45.238749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:58:45.240809 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:45.240809 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:45.240809 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:58:45.240809 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:45.240809 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:58:45.240809 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:45.240809 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:45.240809 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:45.240809 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:45.240809 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:45.240809 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:45.240809 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:45.240809 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:45.240809 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:45.240809 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:45.240809 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:45.240809 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:45.240809 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:45.243464 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:45.243464 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:45.243464 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:58:45.243464 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:58:45.243464 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:58:45.243464 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:58:45.243464 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:58:45.243464 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:58:45.243464 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 15:58:45.243464 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:45.243464 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 314/613] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 15:58:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:45.245361 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6dcd5-522e-433b-bd84-e385322c39fb None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:45.245361 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6dcd5-522e-433b-bd84-e385322c39fb None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:45.245361 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6dcd5-522e-433b-bd84-e385322c39fb None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:45.254071 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 315/614] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:45.254071 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6dcd5-522e-433b-bd84-e385322c39fb None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:45.260182 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 300/615] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 15:58:44 2022] PUT /identity/v3/projects/b1177a85a0de4276898daf4a24ff069f/users/5a84f8aed50c4b3e8e5b33c5b587b26e/roles/cfc67444e9944ffc87173be8c9245108 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:58:45.262675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731afd29-b546-40f5-a6aa-f6b6f08ff321 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:45.263432 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731afd29-b546-40f5-a6aa-f6b6f08ff321 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:45.264734 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731afd29-b546-40f5-a6aa-f6b6f08ff321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:45.302461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-731afd29-b546-40f5-a6aa-f6b6f08ff321 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:45.305029 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-731afd29-b546-40f5-a6aa-f6b6f08ff321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:45.435229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 301/616] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:45.493362 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6dcd5-522e-433b-bd84-e385322c39fb None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:45.500754 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731afd29-b546-40f5-a6aa-f6b6f08ff321 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:45.532708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a9e10-ece7-4a93-898c-f4312c81294d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:45.533398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a9e10-ece7-4a93-898c-f4312c81294d None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:45.533689 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a9e10-ece7-4a93-898c-f4312c81294d None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:45.533993 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a9e10-ece7-4a93-898c-f4312c81294d None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:45.534849 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a9e10-ece7-4a93-898c-f4312c81294d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:45.535170 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a9e10-ece7-4a93-898c-f4312c81294d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:45.536686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a9e10-ece7-4a93-898c-f4312c81294d None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:45.555578 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:45.556604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 316/617] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jun 16 15:58:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:45.569960 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a9e10-ece7-4a93-898c-f4312c81294d None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:45.604695 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:45.649293 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731afd29-b546-40f5-a6aa-f6b6f08ff321 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:45.681897 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a9e10-ece7-4a93-898c-f4312c81294d None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:45.686586 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1697d66d-2e40-4502-aad7-407276c001e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:45.687655 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697d66d-2e40-4502-aad7-407276c001e8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:45.688163 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697d66d-2e40-4502-aad7-407276c001e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:45.688638 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697d66d-2e40-4502-aad7-407276c001e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:45.709273 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1697d66d-2e40-4502-aad7-407276c001e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:45.720114 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-1697d66d-2e40-4502-aad7-407276c001e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:45.720114 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:45.720114 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:45.720114 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:45.720114 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:45.720114 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:45.720114 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:45.720114 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:45.720114 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:45.720114 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:45.720114 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:45.720114 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:45.720114 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:45.720114 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:45.720114 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:45.720114 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:45.721116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:45.721116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:45.721116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:45.721116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:45.721116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:45.721116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:45.721116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:45.721116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:45.721116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:45.721116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:45.721116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:45.721116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:45.721116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:45.721116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:45.721116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:45.721116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:45.722002 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:45.722002 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:45.722002 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:45.722002 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:45.722002 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:45.722002 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:45.722002 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:45.722002 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:45.722002 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:45.722002 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:45.722002 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:45.722002 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:45.724783 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 302/618] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 15:58:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:45.730298 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1697d66d-2e40-4502-aad7-407276c001e8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:45.777862 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4e9bc-fdba-4457-be5b-365f249cee43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:45.778689 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4e9bc-fdba-4457-be5b-365f249cee43 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:45.778971 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4e9bc-fdba-4457-be5b-365f249cee43 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:45.779256 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4e9bc-fdba-4457-be5b-365f249cee43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:45.819081 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4e9bc-fdba-4457-be5b-365f249cee43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:45.821287 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4e9bc-fdba-4457-be5b-365f249cee43 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:45.850277 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1697d66d-2e40-4502-aad7-407276c001e8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:45.866034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-83b4e9bc-fdba-4457-be5b-365f249cee43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:45.866034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:45.866034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:45.866034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:45.866034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:45.866034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:45.866034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:45.866034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:45.866034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:45.866034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:45.866034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:45.866034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:45.866034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:45.866034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:45.866034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:45.867143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:45.867143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:45.867143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:45.867143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:45.867143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:45.867143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:45.867143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:45.867143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:45.867143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:45.867143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:45.867143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:45.867143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:45.867143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:45.867143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:45.867143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:45.867143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:45.867143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:45.868043 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:45.868043 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:45.868043 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:45.868043 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:45.868269 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:45.868342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 317/619] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:45.874934 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4e9bc-fdba-4457-be5b-365f249cee43 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:45.893272 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacdb9a-0471-42da-8640-c00484387030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:45.894174 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacdb9a-0471-42da-8640-c00484387030 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:45.894584 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacdb9a-0471-42da-8640-c00484387030 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:45.894963 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacdb9a-0471-42da-8640-c00484387030 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:45.895827 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acacdb9a-0471-42da-8640-c00484387030 None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:45.896240 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acacdb9a-0471-42da-8640-c00484387030 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:45.897721 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acacdb9a-0471-42da-8640-c00484387030 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:45.906931 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 303/620] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 15:58:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:45.915294 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacdb9a-0471-42da-8640-c00484387030 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:46.049204 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:46.072308 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4e9bc-fdba-4457-be5b-365f249cee43 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:46.098423 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacdb9a-0471-42da-8640-c00484387030 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:46.135512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d945739-e150-4b24-bbed-5d1f9537fa85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:46.136179 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d945739-e150-4b24-bbed-5d1f9537fa85 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:46.136460 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d945739-e150-4b24-bbed-5d1f9537fa85 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:46.136729 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d945739-e150-4b24-bbed-5d1f9537fa85 None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:46.137444 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d945739-e150-4b24-bbed-5d1f9537fa85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:46.137745 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d945739-e150-4b24-bbed-5d1f9537fa85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:46.139289 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d945739-e150-4b24-bbed-5d1f9537fa85 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:46.160171 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 318/621] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:46.162374 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe780e98-4884-4805-8ebd-fd81781da4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:46.163108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe780e98-4884-4805-8ebd-fd81781da4c0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:46.163385 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe780e98-4884-4805-8ebd-fd81781da4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:46.163657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe780e98-4884-4805-8ebd-fd81781da4c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:46.166591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d945739-e150-4b24-bbed-5d1f9537fa85 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:46.188405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe780e98-4884-4805-8ebd-fd81781da4c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:46.189900 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe780e98-4884-4805-8ebd-fd81781da4c0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:46.210695 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-fe780e98-4884-4805-8ebd-fd81781da4c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:46.210695 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:46.210695 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:46.210695 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:46.210695 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:46.210695 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:46.210695 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:46.210695 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:46.210695 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:46.210695 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:46.210695 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:46.210695 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:46.210695 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:46.210695 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:46.210695 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:46.211863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:46.211863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:46.211863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:46.211863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:46.211863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:46.211863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:46.211863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:46.211863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:46.211863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:46.211863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:46.211863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:46.211863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:46.211863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:46.211863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:46.211863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:46.211863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:46.211863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:46.212769 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:46.212769 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:46.212769 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:46.212769 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:46.216826 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 304/622] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:46.218366 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe780e98-4884-4805-8ebd-fd81781da4c0 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:46.234293 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:46.365521 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d945739-e150-4b24-bbed-5d1f9537fa85 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:46.387931 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe780e98-4884-4805-8ebd-fd81781da4c0 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:46.403692 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c036234-0378-4951-973a-2d53e8d29729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:46.404616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c036234-0378-4951-973a-2d53e8d29729 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:46.405036 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c036234-0378-4951-973a-2d53e8d29729 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:46.405512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c036234-0378-4951-973a-2d53e8d29729 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:46.413012 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c036234-0378-4951-973a-2d53e8d29729 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:46.414603 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c036234-0378-4951-973a-2d53e8d29729 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:46.423645 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d6c0d-1f69-4ec9-a816-9c185a6482cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:46.424312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d6c0d-1f69-4ec9-a816-9c185a6482cc None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:46.424599 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d6c0d-1f69-4ec9-a816-9c185a6482cc None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:46.424877 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d6c0d-1f69-4ec9-a816-9c185a6482cc None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:46.425601 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d6c0d-1f69-4ec9-a816-9c185a6482cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:46.425913 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d6c0d-1f69-4ec9-a816-9c185a6482cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:46.427344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d6c0d-1f69-4ec9-a816-9c185a6482cc None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:46.430851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-3c036234-0378-4951-973a-2d53e8d29729 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:46.430851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:46.430851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:46.430851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:46.430851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:46.430851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:46.430851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:46.430851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:46.430851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:46.430851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:46.430851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:46.430851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:46.430851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:58:46.430851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:58:46.430851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:58:46.432044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:58:46.432044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:46.432044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:46.432044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:46.432044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:46.432044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:46.432044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:46.432044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:46.432044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:46.432044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:46.432044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:46.432044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:46.432044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:46.432044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:46.432044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:46.432044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:46.432044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:46.432939 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:58:46.432939 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:58:46.432939 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:58:46.432939 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:58:46.432939 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:58:46.432939 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:58:46.432939 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:46.432939 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:46.433379 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 319/623] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:46.436389 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398059fe-f713-4e1e-a9a2-250402389081 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:46.436609 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398059fe-f713-4e1e-a9a2-250402389081 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:46.438025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398059fe-f713-4e1e-a9a2-250402389081 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:46.442529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 320/624] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:46.443790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9ccf3-647b-4645-9296-34057dfd26ee None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:46.444151 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9ccf3-647b-4645-9296-34057dfd26ee None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:46.444329 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9ccf3-647b-4645-9296-34057dfd26ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:46.447879 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 305/625] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:46.448743 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d6c0d-1f69-4ec9-a816-9c185a6482cc None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:46.463235 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-b2c9ccf3-647b-4645-9296-34057dfd26ee None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:46.464903 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-b2c9ccf3-647b-4645-9296-34057dfd26ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:46.467250 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:46.523086 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:46.556539 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d6c0d-1f69-4ec9-a816-9c185a6482cc None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:46.557073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 321/626] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:46.558449 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c9ccf3-647b-4645-9296-34057dfd26ee None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:46.593079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489fd7e-a79c-41bc-9f47-77836be998f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:46.593753 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489fd7e-a79c-41bc-9f47-77836be998f4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:46.597982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489fd7e-a79c-41bc-9f47-77836be998f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:46.597982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489fd7e-a79c-41bc-9f47-77836be998f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:46.597982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489fd7e-a79c-41bc-9f47-77836be998f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:46.597982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489fd7e-a79c-41bc-9f47-77836be998f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:46.597982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489fd7e-a79c-41bc-9f47-77836be998f4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:46.611910 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 306/627] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:58:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:46.615179 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489fd7e-a79c-41bc-9f47-77836be998f4 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:46.695370 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c9ccf3-647b-4645-9296-34057dfd26ee None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:46.721958 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489fd7e-a79c-41bc-9f47-77836be998f4 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:46.735713 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292c78c-30d0-4e52-a971-9cf7ed3884b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:46.737323 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292c78c-30d0-4e52-a971-9cf7ed3884b0 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:46.738084 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292c78c-30d0-4e52-a971-9cf7ed3884b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:46.738568 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292c78c-30d0-4e52-a971-9cf7ed3884b0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:46.749228 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b292c78c-30d0-4e52-a971-9cf7ed3884b0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:46.751633 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b292c78c-30d0-4e52-a971-9cf7ed3884b0 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:46.754659 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:46.755426 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:46.761667 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37683149-8cbc-48cf-b750-8f801ac0b837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:46.762509 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37683149-8cbc-48cf-b750-8f801ac0b837 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:46.762793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37683149-8cbc-48cf-b750-8f801ac0b837 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:46.763074 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37683149-8cbc-48cf-b750-8f801ac0b837 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:46.774329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37683149-8cbc-48cf-b750-8f801ac0b837 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:46.785121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-37683149-8cbc-48cf-b750-8f801ac0b837 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:46.785121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:46.785121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:46.785121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:46.785121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:46.785121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:46.785121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:46.785121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:46.785121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:46.785121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:46.785121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:46.785121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:46.785121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:46.785121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:46.785121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:46.785121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:46.786141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:46.786141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:46.786141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:46.786141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:46.786141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:46.786141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:46.786141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:46.786141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:46.786141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:46.786141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:46.786141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:46.786141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:46.786141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:46.786141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:46.786141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:46.786141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:46.786992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:46.786992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:46.786992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:46.786992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:46.786992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:46.786992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:46.786992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:46.786992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:46.786992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:46.786992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:46.786992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:46.786992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:46.789228 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 307/628] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 15:58:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:46.790107 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37683149-8cbc-48cf-b750-8f801ac0b837 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:46.791932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-b292c78c-30d0-4e52-a971-9cf7ed3884b0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 15:58:46.791932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:46.791932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:46.791932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:46.791932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:46.791932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:46.791932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:46.791932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:46.791932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:46.791932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:46.791932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:46.791932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:46.791932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:58:46.791932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:58:46.791932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:58:46.791932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:58:46.792902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:46.792902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:46.792902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:58:46.792902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:46.792902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:58:46.792902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:46.792902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:46.792902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:46.792902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:46.792902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:46.792902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:46.792902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:46.792902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:46.792902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:46.792902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:46.792902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:46.792902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:46.792902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:46.793841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:46.793841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:46.793841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:58:46.793841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:58:46.793841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:58:46.793841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:58:46.793841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:58:46.793841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:58:46.793841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 15:58:46.793841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:46.795922 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 322/629] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jun 16 15:58:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:46.802173 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292c78c-30d0-4e52-a971-9cf7ed3884b0 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:46.918133 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37683149-8cbc-48cf-b750-8f801ac0b837 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:46.926464 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292c78c-30d0-4e52-a971-9cf7ed3884b0 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:46.963018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf99da9-f848-49ec-afe4-2117d42a0cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:46.963614 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf99da9-f848-49ec-afe4-2117d42a0cdc None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:46.963955 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf99da9-f848-49ec-afe4-2117d42a0cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:46.964239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf99da9-f848-49ec-afe4-2117d42a0cdc None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:46.964878 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398007e7-fd45-4c01-9418-c073d1fb7159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:46.965000 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf99da9-f848-49ec-afe4-2117d42a0cdc None admin] RBAC: Adding query filter params (name=service) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:46.965289 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf99da9-f848-49ec-afe4-2117d42a0cdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:46.966352 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398007e7-fd45-4c01-9418-c073d1fb7159 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:46.966717 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf99da9-f848-49ec-afe4-2117d42a0cdc None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:46.967074 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398007e7-fd45-4c01-9418-c073d1fb7159 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:46.967561 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398007e7-fd45-4c01-9418-c073d1fb7159 None admin] PATH_INFO: `/v3/projects/b1177a85a0de4276898daf4a24ff069f/users/0082e37ce54f49f7837601349508c89f/roles/cfc67444e9944ffc87173be8c9245108` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:46.980883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 323/630] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 15:58:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:46.981855 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf99da9-f848-49ec-afe4-2117d42a0cdc None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:46.988477 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398007e7-fd45-4c01-9418-c073d1fb7159 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1177a85a0de4276898daf4a24ff069f, user_id=0082e37ce54f49f7837601349508c89f, role_id=cfc67444e9944ffc87173be8c9245108)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:46.990418 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398007e7-fd45-4c01-9418-c073d1fb7159 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:47.008455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 308/631] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 15:58:46 2022] PUT /identity/v3/projects/b1177a85a0de4276898daf4a24ff069f/users/0082e37ce54f49f7837601349508c89f/roles/cfc67444e9944ffc87173be8c9245108 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:58:47.009992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398007e7-fd45-4c01-9418-c073d1fb7159 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:47.069459 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:47.089753 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:47.151856 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398007e7-fd45-4c01-9418-c073d1fb7159 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:47.190560 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf99da9-f848-49ec-afe4-2117d42a0cdc None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:47.198469 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1532195-fd5d-4503-bbf4-b70a50b40a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:47.199664 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1532195-fd5d-4503-bbf4-b70a50b40a5c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:47.200183 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1532195-fd5d-4503-bbf4-b70a50b40a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:47.200675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1532195-fd5d-4503-bbf4-b70a50b40a5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:47.215269 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1532195-fd5d-4503-bbf4-b70a50b40a5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:47.218148 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1532195-fd5d-4503-bbf4-b70a50b40a5c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:47.242185 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-c1532195-fd5d-4503-bbf4-b70a50b40a5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:47.242185 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:47.242185 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:47.242185 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:47.242185 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:47.242185 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:47.242185 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:47.242185 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:47.242185 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:47.242185 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:47.242185 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:47.242185 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:47.242185 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:47.242185 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:47.242185 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:47.243152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:47.243152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:47.243152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:47.243152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:47.243152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:47.243152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:47.243152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:47.243152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:47.243152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:47.243152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:47.243152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:47.243152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:47.243152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:47.243152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:47.243152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:47.243152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:47.243152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:47.244054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:47.244054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:47.244054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:47.244054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:47.246823 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 309/632] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:47.255498 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1532195-fd5d-4503-bbf4-b70a50b40a5c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:47.276529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15d93e-0f53-4766-9209-5059b44b0265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:47.277285 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15d93e-0f53-4766-9209-5059b44b0265 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:47.277569 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15d93e-0f53-4766-9209-5059b44b0265 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:47.277844 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15d93e-0f53-4766-9209-5059b44b0265 None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:47.278793 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b15d93e-0f53-4766-9209-5059b44b0265 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:47.279115 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b15d93e-0f53-4766-9209-5059b44b0265 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:47.282679 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b15d93e-0f53-4766-9209-5059b44b0265 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:47.308841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 324/633] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jun 16 15:58:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:47.310182 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d2bb4-0137-459d-8e99-4be3a05fda9e None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:47.310477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d2bb4-0137-459d-8e99-4be3a05fda9e None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:47.310898 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d2bb4-0137-459d-8e99-4be3a05fda9e None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:47.313178 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 325/634] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:47.318960 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941f664-5db8-4e98-9aca-9a03751a0198 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:47.319288 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941f664-5db8-4e98-9aca-9a03751a0198 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:47.320326 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941f664-5db8-4e98-9aca-9a03751a0198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:47.339492 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-b941f664-5db8-4e98-9aca-9a03751a0198 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:47.341082 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-b941f664-5db8-4e98-9aca-9a03751a0198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:47.408797 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:47.418249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1532195-fd5d-4503-bbf4-b70a50b40a5c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:47.426024 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 326/635] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:47.427151 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941f664-5db8-4e98-9aca-9a03751a0198 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:47.470717 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0598d5-5de0-45ab-8073-9b5676cd5081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:47.471467 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0598d5-5de0-45ab-8073-9b5676cd5081 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:47.471746 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0598d5-5de0-45ab-8073-9b5676cd5081 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:47.472022 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0598d5-5de0-45ab-8073-9b5676cd5081 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:47.472754 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0598d5-5de0-45ab-8073-9b5676cd5081 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:47.473057 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0598d5-5de0-45ab-8073-9b5676cd5081 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:47.474641 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0598d5-5de0-45ab-8073-9b5676cd5081 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:47.490503 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 310/636] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:47.491371 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0598d5-5de0-45ab-8073-9b5676cd5081 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:47.493069 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:47.554662 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941f664-5db8-4e98-9aca-9a03751a0198 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:47.594339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52401c65-70f3-4609-ae33-89898ec4cfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:47.595373 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52401c65-70f3-4609-ae33-89898ec4cfc2 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:47.595890 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52401c65-70f3-4609-ae33-89898ec4cfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:47.596392 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52401c65-70f3-4609-ae33-89898ec4cfc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:47.618561 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0598d5-5de0-45ab-8073-9b5676cd5081 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:47.659495 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab56086-1bc6-4aa1-bf84-da6ac79d420b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:47.660403 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab56086-1bc6-4aa1-bf84-da6ac79d420b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:47.660949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab56086-1bc6-4aa1-bf84-da6ac79d420b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:47.660949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab56086-1bc6-4aa1-bf84-da6ac79d420b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:47.671418 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52401c65-70f3-4609-ae33-89898ec4cfc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:47.672613 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab56086-1bc6-4aa1-bf84-da6ac79d420b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:47.672922 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52401c65-70f3-4609-ae33-89898ec4cfc2 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:47.680315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-1ab56086-1bc6-4aa1-bf84-da6ac79d420b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:47.680315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:47.680315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:47.680315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:47.680315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:47.680315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:47.680315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:47.680315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:47.680315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:47.680315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:47.680315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:47.680315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:47.680315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:47.680315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:47.680315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:47.680315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:47.681302 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:47.681302 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:47.681302 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:47.681302 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:47.681302 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:47.681302 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:47.681302 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:47.681302 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:47.681302 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:47.681302 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:47.681302 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:47.681302 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:47.681302 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:47.681302 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:47.681302 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:47.681302 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:47.683011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:47.683011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:47.683011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:47.683011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:47.683011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:47.683011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:47.683011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:47.683011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:47.683011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:47.683011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:47.683011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:47.683011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:47.683011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 311/637] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 15:58:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:47.683011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab56086-1bc6-4aa1-bf84-da6ac79d420b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:47.702764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:47.732121 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:47.790250 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 327/638] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:47.791731 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31b71b-e662-4f93-ad6d-f7807a5ffbf1 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:47.792076 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31b71b-e662-4f93-ad6d-f7807a5ffbf1 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:47.792356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31b71b-e662-4f93-ad6d-f7807a5ffbf1 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:47.797998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 328/639] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:47.797998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31b71b-e662-4f93-ad6d-f7807a5ffbf1 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:47.811826 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab56086-1bc6-4aa1-bf84-da6ac79d420b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:47.849164 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9916ff4-c954-4755-a0d8-9e19e8eddd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:47.849991 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9916ff4-c954-4755-a0d8-9e19e8eddd66 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:47.850279 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9916ff4-c954-4755-a0d8-9e19e8eddd66 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:47.850557 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9916ff4-c954-4755-a0d8-9e19e8eddd66 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:47.859231 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9916ff4-c954-4755-a0d8-9e19e8eddd66 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:47.860817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9916ff4-c954-4755-a0d8-9e19e8eddd66 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:47.875488 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-d9916ff4-c954-4755-a0d8-9e19e8eddd66 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 15:58:47.875488 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:47.875488 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:47.875488 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:47.875488 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:47.875488 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:47.875488 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:47.875488 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:47.875488 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:47.875488 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:47.875488 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:47.875488 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:47.875488 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:58:47.875488 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:58:47.875488 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:58:47.875488 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:58:47.876500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:47.876500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:47.876500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:58:47.876500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:47.876500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:58:47.876500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:47.876500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:47.876500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:47.876500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:47.876500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:47.876500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:47.876500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:47.876500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:47.876500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:47.876500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:47.876500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:47.876500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:47.876500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:47.877443 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:47.877443 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:47.877443 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:58:47.877443 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:58:47.877443 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:58:47.877443 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:58:47.877443 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:58:47.877443 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:58:47.877443 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 15:58:47.877443 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:47.877443 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 312/640] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:47.878069 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9916ff4-c954-4755-a0d8-9e19e8eddd66 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:47.912652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31b71b-e662-4f93-ad6d-f7807a5ffbf1 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:47.948892 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8207c5f2-fcf6-4b58-ac62-87c5d988b6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:47.949568 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207c5f2-fcf6-4b58-ac62-87c5d988b6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:47.949850 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207c5f2-fcf6-4b58-ac62-87c5d988b6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:47.950167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207c5f2-fcf6-4b58-ac62-87c5d988b6b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:47.950918 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8207c5f2-fcf6-4b58-ac62-87c5d988b6b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:47.951229 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8207c5f2-fcf6-4b58-ac62-87c5d988b6b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:47.953139 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8207c5f2-fcf6-4b58-ac62-87c5d988b6b8 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:47.966241 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 329/641] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 15:58:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:47.966241 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387997d7-99f4-4705-89f4-f27c48aa2dba None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:47.966241 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387997d7-99f4-4705-89f4-f27c48aa2dba None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:47.966241 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387997d7-99f4-4705-89f4-f27c48aa2dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:47.968553 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:47.982898 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-387997d7-99f4-4705-89f4-f27c48aa2dba None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:47.984562 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-387997d7-99f4-4705-89f4-f27c48aa2dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:47.987713 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:47.997485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9916ff4-c954-4755-a0d8-9e19e8eddd66 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:48.035125 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b07d94-fb93-410f-b930-64edf4515ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:48.035921 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b07d94-fb93-410f-b930-64edf4515ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:48.036554 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b07d94-fb93-410f-b930-64edf4515ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:48.036846 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b07d94-fb93-410f-b930-64edf4515ca8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:48.049653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b07d94-fb93-410f-b930-64edf4515ca8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:48.051255 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b07d94-fb93-410f-b930-64edf4515ca8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:48.066978 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 330/642] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:48.068229 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387997d7-99f4-4705-89f4-f27c48aa2dba None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:48.070970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-d4b07d94-fb93-410f-b930-64edf4515ca8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:48.070970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:48.070970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:48.070970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:48.070970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:48.070970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:48.070970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:48.070970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:48.070970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:48.070970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:48.070970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:48.070970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:48.070970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:48.070970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:48.070970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:48.071924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:48.071924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:48.071924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:48.071924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:48.071924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:48.071924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:48.071924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:48.071924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:48.071924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:48.071924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:48.071924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:48.071924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:48.071924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:48.071924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:48.071924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:48.071924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:48.071924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:48.072842 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:48.072842 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:48.072842 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:48.072842 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:48.072842 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 313/643] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:48.073334 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b07d94-fb93-410f-b930-64edf4515ca8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:48.204746 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b07d94-fb93-410f-b930-64edf4515ca8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:48.214869 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387997d7-99f4-4705-89f4-f27c48aa2dba None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:48.240287 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:48.251188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97cb0c4-9e4b-4ea6-af21-acf4eec0fbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:48.251982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97cb0c4-9e4b-4ea6-af21-acf4eec0fbfc None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:48.252267 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97cb0c4-9e4b-4ea6-af21-acf4eec0fbfc None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:48.252541 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97cb0c4-9e4b-4ea6-af21-acf4eec0fbfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:48.252849 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f262e6-df77-4200-9897-feaf59e890fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:48.253690 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f262e6-df77-4200-9897-feaf59e890fd None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:48.254005 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f262e6-df77-4200-9897-feaf59e890fd None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:48.254508 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f262e6-df77-4200-9897-feaf59e890fd None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:48.255477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f262e6-df77-4200-9897-feaf59e890fd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:48.255790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f262e6-df77-4200-9897-feaf59e890fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:48.257262 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f262e6-df77-4200-9897-feaf59e890fd None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:48.268565 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:48.272539 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97cb0c4-9e4b-4ea6-af21-acf4eec0fbfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:48.274076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97cb0c4-9e4b-4ea6-af21-acf4eec0fbfc None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:48.285535 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 331/644] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jun 16 15:58:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:48.285535 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f262e6-df77-4200-9897-feaf59e890fd None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:48.298685 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-e97cb0c4-9e4b-4ea6-af21-acf4eec0fbfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:48.298685 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:48.298685 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:48.298685 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:48.298685 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:48.298685 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:48.298685 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:48.298685 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:48.298685 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:48.298685 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:48.298685 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:48.298685 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:48.298685 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:48.298685 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:48.298685 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:48.299698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:48.299698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:48.299698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:48.299698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:48.299698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:48.299698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:48.299698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:48.299698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:48.299698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:48.299698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:48.299698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:48.299698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:48.299698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:48.299698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:48.299698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:48.299698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:48.299698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:48.305401 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:48.305401 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:48.305401 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:48.305401 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:48.305401 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 314/645] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:48.305401 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97cb0c4-9e4b-4ea6-af21-acf4eec0fbfc None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:48.412101 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f262e6-df77-4200-9897-feaf59e890fd None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:48.432460 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97cb0c4-9e4b-4ea6-af21-acf4eec0fbfc None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:48.450323 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c1e41-36e0-466a-acda-9ce76cb9f96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:48.451295 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c1e41-36e0-466a-acda-9ce76cb9f96c None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:48.452335 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c1e41-36e0-466a-acda-9ce76cb9f96c None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:48.452756 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c1e41-36e0-466a-acda-9ce76cb9f96c None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:48.453620 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c1e41-36e0-466a-acda-9ce76cb9f96c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:48.454126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c1e41-36e0-466a-acda-9ce76cb9f96c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:48.455665 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c1e41-36e0-466a-acda-9ce76cb9f96c None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:48.468540 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 332/646] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:48.469369 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c1e41-36e0-466a-acda-9ce76cb9f96c None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:48.480275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f415d3f-0b1d-4ce8-ba6f-c84b016af634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:48.481076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f415d3f-0b1d-4ce8-ba6f-c84b016af634 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:48.481355 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f415d3f-0b1d-4ce8-ba6f-c84b016af634 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:48.481626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f415d3f-0b1d-4ce8-ba6f-c84b016af634 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:48.493672 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f415d3f-0b1d-4ce8-ba6f-c84b016af634 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:48.502770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-1f415d3f-0b1d-4ce8-ba6f-c84b016af634 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:48.502770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:48.502770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:48.502770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:48.502770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:48.502770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:48.502770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:48.502770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:48.502770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:48.502770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:48.502770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:48.502770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:48.502770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:48.502770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:48.502770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:48.502770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:48.503986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:48.503986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:48.503986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:48.503986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:48.503986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:48.503986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:48.503986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:48.503986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:48.503986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:48.503986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:48.503986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:48.503986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:48.503986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:48.503986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:48.503986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:48.503986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:48.504967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:48.504967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:48.504967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:48.504967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:48.504967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:48.504967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:48.504967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:48.504967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:48.504967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:48.504967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:48.504967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:48.504967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:48.504967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 315/647] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 15:58:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:48.505779 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f415d3f-0b1d-4ce8-ba6f-c84b016af634 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:48.544150 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:48.551594 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:48.599963 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c1e41-36e0-466a-acda-9ce76cb9f96c None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:48.628293 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f415d3f-0b1d-4ce8-ba6f-c84b016af634 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:48.640520 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311087e0-61ce-4c2e-8278-56c6484a63f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:48.641345 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311087e0-61ce-4c2e-8278-56c6484a63f7 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:48.641702 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311087e0-61ce-4c2e-8278-56c6484a63f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:48.642072 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311087e0-61ce-4c2e-8278-56c6484a63f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:48.664370 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fdcc8-f6ca-44fc-8ae2-797cf226742f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:48.665029 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fdcc8-f6ca-44fc-8ae2-797cf226742f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:48.665318 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fdcc8-f6ca-44fc-8ae2-797cf226742f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:48.665586 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fdcc8-f6ca-44fc-8ae2-797cf226742f None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:48.666328 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319fdcc8-f6ca-44fc-8ae2-797cf226742f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:48.666634 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319fdcc8-f6ca-44fc-8ae2-797cf226742f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:48.667997 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319fdcc8-f6ca-44fc-8ae2-797cf226742f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:48.684273 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 316/648] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:48.684273 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fdcc8-f6ca-44fc-8ae2-797cf226742f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:48.722996 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311087e0-61ce-4c2e-8278-56c6484a63f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:48.724763 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311087e0-61ce-4c2e-8278-56c6484a63f7 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:48.787026 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:48.791268 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fdcc8-f6ca-44fc-8ae2-797cf226742f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:48.801801 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:48.831259 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad631bce-50cd-41f5-bdb8-0630d19c1680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:48.832062 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad631bce-50cd-41f5-bdb8-0630d19c1680 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:48.832343 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad631bce-50cd-41f5-bdb8-0630d19c1680 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:48.832623 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad631bce-50cd-41f5-bdb8-0630d19c1680 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:48.839679 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad631bce-50cd-41f5-bdb8-0630d19c1680 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:48.841108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad631bce-50cd-41f5-bdb8-0630d19c1680 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:48.856076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-ad631bce-50cd-41f5-bdb8-0630d19c1680 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:48.856076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:48.856076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:48.856076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:48.856076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:48.856076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:48.856076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:48.856076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:48.856076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:48.856076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:48.856076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:48.856076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:48.856076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:58:48.856076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:58:48.856076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:58:48.857038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:58:48.857038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:48.857038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:48.857038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:48.857038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:48.857038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:48.857038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:48.857038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:48.857038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:48.857038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:48.857038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:48.857038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:48.857038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:48.857038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:48.857038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:48.857038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:48.857038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:48.858291 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:58:48.858291 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:58:48.858291 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:58:48.858291 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:58:48.858291 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:58:48.858291 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:58:48.858291 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:48.858291 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:48.858291 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 317/649] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:48.858771 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad631bce-50cd-41f5-bdb8-0630d19c1680 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:48.860998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 333/650] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:48.862288 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311087e0-61ce-4c2e-8278-56c6484a63f7 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:48.975128 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311087e0-61ce-4c2e-8278-56c6484a63f7 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:48.978523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad631bce-50cd-41f5-bdb8-0630d19c1680 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:49.013590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acdb4f9-2021-4688-ab6b-69ad7a2e56ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:49.015124 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acdb4f9-2021-4688-ab6b-69ad7a2e56ae None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:49.015409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acdb4f9-2021-4688-ab6b-69ad7a2e56ae None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:49.015796 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acdb4f9-2021-4688-ab6b-69ad7a2e56ae None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:49.017882 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fa72e-9343-4865-a797-60e154fc9a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:49.018595 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fa72e-9343-4865-a797-60e154fc9a81 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:49.018881 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fa72e-9343-4865-a797-60e154fc9a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:49.019165 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fa72e-9343-4865-a797-60e154fc9a81 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:49.019917 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fa72e-9343-4865-a797-60e154fc9a81 None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:49.020229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fa72e-9343-4865-a797-60e154fc9a81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:49.021614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fa72e-9343-4865-a797-60e154fc9a81 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:49.024486 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acdb4f9-2021-4688-ab6b-69ad7a2e56ae None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:49.026191 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acdb4f9-2021-4688-ab6b-69ad7a2e56ae None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:49.031206 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 318/651] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 15:58:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:49.032034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fa72e-9343-4865-a797-60e154fc9a81 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:49.042952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-6acdb4f9-2021-4688-ab6b-69ad7a2e56ae None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 15:58:49.042952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:49.042952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:49.042952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:49.042952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:49.042952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:49.042952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:49.042952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:49.042952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:49.042952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:49.042952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:49.042952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:49.042952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:58:49.042952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:58:49.042952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:58:49.042952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:58:49.043873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:49.043873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:49.043873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:58:49.043873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:49.043873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:58:49.043873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:49.043873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:49.043873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:49.043873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:49.043873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:49.043873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:49.043873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:49.043873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:49.043873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:49.043873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:49.043873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:49.043873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:49.043873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:49.044803 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:49.044803 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:49.044803 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:58:49.044803 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:58:49.044803 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:58:49.044803 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:58:49.044803 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:58:49.044803 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:58:49.044803 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 15:58:49.044803 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:49.047114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 334/652] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:49.048156 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acdb4f9-2021-4688-ab6b-69ad7a2e56ae None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:49.057982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:49.071893 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:49.150666 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fa72e-9343-4865-a797-60e154fc9a81 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:49.169318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acdb4f9-2021-4688-ab6b-69ad7a2e56ae None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:49.188669 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7694e5-b32e-4dcc-8572-957985fff324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:49.189640 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7694e5-b32e-4dcc-8572-957985fff324 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:49.189945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7694e5-b32e-4dcc-8572-957985fff324 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:49.190227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7694e5-b32e-4dcc-8572-957985fff324 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:49.191022 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7694e5-b32e-4dcc-8572-957985fff324 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:49.191361 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7694e5-b32e-4dcc-8572-957985fff324 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:49.193329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7694e5-b32e-4dcc-8572-957985fff324 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:49.208148 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 319/653] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:58:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:49.208761 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4db02-4116-4687-aa2a-6a13716d3010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:49.209411 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7694e5-b32e-4dcc-8572-957985fff324 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:49.209892 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4db02-4116-4687-aa2a-6a13716d3010 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:49.210212 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4db02-4116-4687-aa2a-6a13716d3010 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:49.210493 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4db02-4116-4687-aa2a-6a13716d3010 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:49.225071 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce4db02-4116-4687-aa2a-6a13716d3010 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:49.226696 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce4db02-4116-4687-aa2a-6a13716d3010 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:49.246824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-bce4db02-4116-4687-aa2a-6a13716d3010 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:49.246824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:49.246824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:49.246824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:49.246824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:49.246824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:49.246824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:49.246824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:49.246824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:49.246824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:49.246824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:49.246824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:49.246824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:49.246824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:49.246824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:49.247791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:49.247791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:49.247791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:49.247791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:49.247791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:49.247791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:49.247791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:49.247791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:49.247791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:49.247791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:49.247791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:49.247791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:49.247791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:49.247791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:49.247791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:49.247791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:49.247791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:49.254121 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:49.254121 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:49.254121 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:49.254121 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:49.254121 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 335/654] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:49.254121 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4db02-4116-4687-aa2a-6a13716d3010 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:49.320033 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:49.331699 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:49.346707 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7694e5-b32e-4dcc-8572-957985fff324 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:49.367848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4db02-4116-4687-aa2a-6a13716d3010 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:49.383504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92346951-c0cf-45f4-a0b9-7c6946f0fd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:49.384562 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92346951-c0cf-45f4-a0b9-7c6946f0fd68 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:49.384850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92346951-c0cf-45f4-a0b9-7c6946f0fd68 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:49.385131 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92346951-c0cf-45f4-a0b9-7c6946f0fd68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:49.395795 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92346951-c0cf-45f4-a0b9-7c6946f0fd68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:49.397241 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92346951-c0cf-45f4-a0b9-7c6946f0fd68 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:49.404280 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17555891-ba23-45aa-981d-4037f451374d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:49.404973 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17555891-ba23-45aa-981d-4037f451374d None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:49.405256 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17555891-ba23-45aa-981d-4037f451374d None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:49.405535 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17555891-ba23-45aa-981d-4037f451374d None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:49.406406 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17555891-ba23-45aa-981d-4037f451374d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:49.406720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17555891-ba23-45aa-981d-4037f451374d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:49.408202 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17555891-ba23-45aa-981d-4037f451374d None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:49.415424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-92346951-c0cf-45f4-a0b9-7c6946f0fd68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:49.415424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:49.415424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:49.415424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:49.415424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:49.415424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:49.415424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:49.415424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:49.415424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:49.415424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:49.415424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:49.415424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:49.415424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:49.415424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:49.415424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:49.416329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:49.416329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:49.416329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:49.416329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:49.416329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:49.416329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:49.416329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:49.416329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:49.416329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:49.416329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:49.416329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:49.416329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:49.416329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:49.416329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:49.416329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:49.416329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:49.416329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:49.417240 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:49.417240 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:49.417240 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:49.417240 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:49.417240 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 320/655] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:49.418025 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92346951-c0cf-45f4-a0b9-7c6946f0fd68 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:49.431684 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 336/656] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jun 16 15:58:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:49.433072 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17555891-ba23-45aa-981d-4037f451374d None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:49.533712 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92346951-c0cf-45f4-a0b9-7c6946f0fd68 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:49.541584 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17555891-ba23-45aa-981d-4037f451374d None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:49.564706 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:49.570174 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:49.571025 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f6093-702e-4110-acde-86d1f2719218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:49.571598 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f6093-702e-4110-acde-86d1f2719218 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:49.571882 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f6093-702e-4110-acde-86d1f2719218 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:49.572150 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f6093-702e-4110-acde-86d1f2719218 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:49.572973 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f6093-702e-4110-acde-86d1f2719218 None admin] RBAC: Adding query filter params (role.id=cfc67444e9944ffc87173be8c9245108, scope.project.id=b1177a85a0de4276898daf4a24ff069f, user.id=c3645b1a0be34c01ac87c734fcd85ff6) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:49.573281 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f6093-702e-4110-acde-86d1f2719218 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:49.574732 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f6093-702e-4110-acde-86d1f2719218 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:49.579397 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c383dc-cbfa-450c-b41a-bee6e76476e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:49.580049 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c383dc-cbfa-450c-b41a-bee6e76476e4 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:49.580329 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c383dc-cbfa-450c-b41a-bee6e76476e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:49.580600 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c383dc-cbfa-450c-b41a-bee6e76476e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:49.580713 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 321/657] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:58:49 2022] GET /identity/v3/role_assignments?user.id=c3645b1a0be34c01ac87c734fcd85ff6&scope.project.id=b1177a85a0de4276898daf4a24ff069f&role.id=cfc67444e9944ffc87173be8c9245108 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:49.581319 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c383dc-cbfa-450c-b41a-bee6e76476e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:49.581622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c383dc-cbfa-450c-b41a-bee6e76476e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:49.581901 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b6849-7ce5-4d28-8015-08eea5738195 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:49.582217 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b6849-7ce5-4d28-8015-08eea5738195 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:49.582492 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b6849-7ce5-4d28-8015-08eea5738195 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:49.583029 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c383dc-cbfa-450c-b41a-bee6e76476e4 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:49.583924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 322/658] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:49.584392 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b6849-7ce5-4d28-8015-08eea5738195 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:49.602054 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 337/659] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:49.603060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c383dc-cbfa-450c-b41a-bee6e76476e4 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:49.715813 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b6849-7ce5-4d28-8015-08eea5738195 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:49.725440 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c383dc-cbfa-450c-b41a-bee6e76476e4 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:49.763114 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c89224-6fc9-47f2-a013-9dd7a1868ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:49.763904 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c89224-6fc9-47f2-a013-9dd7a1868ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:49.764249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c89224-6fc9-47f2-a013-9dd7a1868ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:49.764600 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c89224-6fc9-47f2-a013-9dd7a1868ba7 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:49.765334 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c89224-6fc9-47f2-a013-9dd7a1868ba7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:49.765648 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c89224-6fc9-47f2-a013-9dd7a1868ba7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:49.766302 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8771ddda-1ee6-4249-bcbf-cb7916117e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:49.767063 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771ddda-1ee6-4249-bcbf-cb7916117e9d None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:49.767197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c89224-6fc9-47f2-a013-9dd7a1868ba7 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:49.767528 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771ddda-1ee6-4249-bcbf-cb7916117e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:49.767900 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771ddda-1ee6-4249-bcbf-cb7916117e9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:49.783201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 323/660] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:49.785485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a337b6-5fb3-4ca5-8e42-7acf380778bd None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:49.786002 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a337b6-5fb3-4ca5-8e42-7acf380778bd None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:49.786431 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a337b6-5fb3-4ca5-8e42-7acf380778bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:49.808467 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-a4a337b6-5fb3-4ca5-8e42-7acf380778bd None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:49.811323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a4a337b6-5fb3-4ca5-8e42-7acf380778bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:49.844573 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:49.860745 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8771ddda-1ee6-4249-bcbf-cb7916117e9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:49.862609 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8771ddda-1ee6-4249-bcbf-cb7916117e9d None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:49.868952 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:49.888030 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 324/661] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:49.888784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a337b6-5fb3-4ca5-8e42-7acf380778bd None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:49.981586 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 338/662] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:49.982517 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8771ddda-1ee6-4249-bcbf-cb7916117e9d None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:50.011220 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a337b6-5fb3-4ca5-8e42-7acf380778bd None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:50.050419 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ed8cb-667d-4bc1-83e4-6c780af32a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:50.051227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ed8cb-667d-4bc1-83e4-6c780af32a07 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:50.051509 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ed8cb-667d-4bc1-83e4-6c780af32a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:50.051787 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ed8cb-667d-4bc1-83e4-6c780af32a07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:50.058964 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ed8cb-667d-4bc1-83e4-6c780af32a07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:50.060418 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ed8cb-667d-4bc1-83e4-6c780af32a07 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:50.074544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-052ed8cb-667d-4bc1-83e4-6c780af32a07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:50.074544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:50.074544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:50.074544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:50.074544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:50.074544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:50.074544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:50.074544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:50.074544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:50.074544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:50.074544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:50.074544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:50.074544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:58:50.074544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:58:50.074544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:58:50.075872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:58:50.075872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:50.075872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:50.075872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:50.075872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:50.075872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:50.075872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:50.075872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:50.075872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:50.075872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:50.075872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:50.075872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:50.075872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:50.075872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:50.075872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:50.075872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:50.075872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:50.077492 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:58:50.077492 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:58:50.077492 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:58:50.077492 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:58:50.077492 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:58:50.077492 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:58:50.077492 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:50.077492 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:50.078909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 325/663] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:50.084337 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ed8cb-667d-4bc1-83e4-6c780af32a07 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:50.084812 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:50.099754 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8771ddda-1ee6-4249-bcbf-cb7916117e9d None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:50.135591 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:50.141071 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabbf447-60bd-4f0d-8c75-532e526dffbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:50.141880 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbf447-60bd-4f0d-8c75-532e526dffbf None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:50.142188 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbf447-60bd-4f0d-8c75-532e526dffbf None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:50.142462 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbf447-60bd-4f0d-8c75-532e526dffbf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:50.150126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabbf447-60bd-4f0d-8c75-532e526dffbf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:50.151587 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabbf447-60bd-4f0d-8c75-532e526dffbf None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:50.167398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-fabbf447-60bd-4f0d-8c75-532e526dffbf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 15:58:50.167398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:50.167398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:50.167398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:50.167398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:50.167398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:50.167398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:50.167398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:50.167398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:50.167398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:50.167398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:50.167398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:50.167398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:58:50.167398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:58:50.167398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:58:50.167398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:58:50.168396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:50.168396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:50.168396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:58:50.168396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:50.168396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:58:50.168396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:50.168396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:50.168396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:50.168396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:50.168396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:50.168396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:50.168396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:50.168396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:50.168396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:50.168396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:50.168396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:50.168396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:50.168396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:50.169423 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:50.169423 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:50.169423 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:58:50.169423 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:58:50.169423 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:58:50.169423 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:58:50.169423 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:58:50.169423 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:58:50.169423 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 15:58:50.169423 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:50.173217 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 339/664] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 15:58:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:50.174106 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabbf447-60bd-4f0d-8c75-532e526dffbf None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:50.196929 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ed8cb-667d-4bc1-83e4-6c780af32a07 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:50.234458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb2435-e9b8-4fac-bd78-993c936b3043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:50.235247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb2435-e9b8-4fac-bd78-993c936b3043 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:50.235529 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb2435-e9b8-4fac-bd78-993c936b3043 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:50.235811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb2435-e9b8-4fac-bd78-993c936b3043 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:50.247059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb2435-e9b8-4fac-bd78-993c936b3043 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:50.254765 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-5bfb2435-e9b8-4fac-bd78-993c936b3043 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:50.254765 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:50.254765 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:50.254765 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:50.254765 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:50.254765 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:50.254765 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:50.254765 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:50.254765 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:50.254765 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:50.254765 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:50.254765 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:50.254765 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:50.254765 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:50.254765 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:50.254765 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:50.255908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:50.255908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:50.255908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:50.255908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:50.255908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:50.255908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:50.255908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:50.255908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:50.255908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:50.255908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:50.255908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:50.255908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:50.255908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:50.255908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:50.255908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:50.255908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:50.257116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:50.257116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:50.257116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:50.257116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:50.257116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:50.257116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:50.257116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:50.257116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:50.257116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:50.257116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:50.257116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:50.257116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:50.261091 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 326/665] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 15:58:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:50.261967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb2435-e9b8-4fac-bd78-993c936b3043 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:50.294110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabbf447-60bd-4f0d-8c75-532e526dffbf None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:50.333761 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d1390a-1d0f-49f7-985c-985dc3539493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:50.334740 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1390a-1d0f-49f7-985c-985dc3539493 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:50.334974 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1390a-1d0f-49f7-985c-985dc3539493 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:50.335247 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1390a-1d0f-49f7-985c-985dc3539493 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:50.346799 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d1390a-1d0f-49f7-985c-985dc3539493 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:50.348258 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d1390a-1d0f-49f7-985c-985dc3539493 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:50.359294 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:50.369792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-a7d1390a-1d0f-49f7-985c-985dc3539493 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:50.369792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:50.369792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:50.369792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:50.369792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:50.369792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:50.369792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:50.369792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:50.369792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:50.369792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:50.369792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:50.369792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:50.369792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:50.369792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:50.369792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:50.370907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:50.370907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:50.370907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:50.370907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:50.370907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:50.370907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:50.370907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:50.370907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:50.370907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:50.370907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:50.370907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:50.370907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:50.370907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:50.370907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:50.370907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:50.370907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:50.370907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:50.371869 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:50.371869 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:50.371869 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:50.371869 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:50.371869 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 340/666] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:50.374174 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d1390a-1d0f-49f7-985c-985dc3539493 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:50.378312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb2435-e9b8-4fac-bd78-993c936b3043 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:50.386590 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:50.416925 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa8107-f436-4559-bbee-84cd2c75aa92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:50.417583 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa8107-f436-4559-bbee-84cd2c75aa92 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:50.417857 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa8107-f436-4559-bbee-84cd2c75aa92 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:50.418175 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa8107-f436-4559-bbee-84cd2c75aa92 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:50.418959 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa8107-f436-4559-bbee-84cd2c75aa92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:50.419269 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa8107-f436-4559-bbee-84cd2c75aa92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:50.421366 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa8107-f436-4559-bbee-84cd2c75aa92 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:50.440502 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 327/667] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:58:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:50.440502 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa8107-f436-4559-bbee-84cd2c75aa92 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:50.500966 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d1390a-1d0f-49f7-985c-985dc3539493 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:50.537225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee8b8e-755a-4602-8512-65bde3d7a46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:50.538318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee8b8e-755a-4602-8512-65bde3d7a46e None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:50.538756 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee8b8e-755a-4602-8512-65bde3d7a46e None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:50.539182 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee8b8e-755a-4602-8512-65bde3d7a46e None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:50.540190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee8b8e-755a-4602-8512-65bde3d7a46e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:50.540675 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee8b8e-755a-4602-8512-65bde3d7a46e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:50.542300 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee8b8e-755a-4602-8512-65bde3d7a46e None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:50.554667 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa8107-f436-4559-bbee-84cd2c75aa92 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:50.563734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 341/668] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jun 16 15:58:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:50.565180 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee8b8e-755a-4602-8512-65bde3d7a46e None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:50.590750 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e1b8d-b641-48ef-b27e-f93f33acb542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:50.591421 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e1b8d-b641-48ef-b27e-f93f33acb542 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:50.591704 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e1b8d-b641-48ef-b27e-f93f33acb542 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:50.592534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e1b8d-b641-48ef-b27e-f93f33acb542 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:50.593977 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e1b8d-b641-48ef-b27e-f93f33acb542 None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:50.594294 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e1b8d-b641-48ef-b27e-f93f33acb542 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:50.595665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e1b8d-b641-48ef-b27e-f93f33acb542 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:50.607288 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 328/669] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 15:58:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:50.607288 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e1b8d-b641-48ef-b27e-f93f33acb542 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:50.641338 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:50.643854 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:50.688113 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee8b8e-755a-4602-8512-65bde3d7a46e None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:50.725725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e1b8d-b641-48ef-b27e-f93f33acb542 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:50.729757 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c568e-1734-43bf-95f8-56f5bba7c76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:50.730543 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c568e-1734-43bf-95f8-56f5bba7c76b None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:50.730799 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c568e-1734-43bf-95f8-56f5bba7c76b None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:50.731070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c568e-1734-43bf-95f8-56f5bba7c76b None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:50.731904 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c568e-1734-43bf-95f8-56f5bba7c76b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:50.732206 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c568e-1734-43bf-95f8-56f5bba7c76b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:50.734197 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c568e-1734-43bf-95f8-56f5bba7c76b None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:50.753629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 342/670] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:50.754684 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c568e-1734-43bf-95f8-56f5bba7c76b None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:50.764293 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895d2d3-364e-4016-9887-0d51b80b7ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:50.765342 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895d2d3-364e-4016-9887-0d51b80b7ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:50.766236 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895d2d3-364e-4016-9887-0d51b80b7ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:50.766540 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895d2d3-364e-4016-9887-0d51b80b7ad1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:50.767381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3895d2d3-364e-4016-9887-0d51b80b7ad1 None admin] RBAC: Adding query filter params (role.id=cfc67444e9944ffc87173be8c9245108, scope.project.id=b1177a85a0de4276898daf4a24ff069f, user.id=6f11beca9c5148eea4cc45f8dc89dbda) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:50.767703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3895d2d3-364e-4016-9887-0d51b80b7ad1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:50.769861 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3895d2d3-364e-4016-9887-0d51b80b7ad1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:50.777570 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 329/671] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:58:50 2022] GET /identity/v3/role_assignments?user.id=6f11beca9c5148eea4cc45f8dc89dbda&scope.project.id=b1177a85a0de4276898daf4a24ff069f&role.id=cfc67444e9944ffc87173be8c9245108 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:50.778437 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895d2d3-364e-4016-9887-0d51b80b7ad1 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:50.896337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c568e-1734-43bf-95f8-56f5bba7c76b None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:50.921198 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:50.928675 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:50.933766 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895d2d3-364e-4016-9887-0d51b80b7ad1 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:50.954951 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1d993-2d18-4b9e-bd77-51ec27f3dd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:50.955613 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1d993-2d18-4b9e-bd77-51ec27f3dd1f None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:50.955971 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1d993-2d18-4b9e-bd77-51ec27f3dd1f None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:50.956328 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1d993-2d18-4b9e-bd77-51ec27f3dd1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:50.986665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874f203-7424-4c0d-924a-0c5d5d10625f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:50.988262 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874f203-7424-4c0d-924a-0c5d5d10625f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:50.988723 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874f203-7424-4c0d-924a-0c5d5d10625f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:50.989187 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874f203-7424-4c0d-924a-0c5d5d10625f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:51.003281 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5874f203-7424-4c0d-924a-0c5d5d10625f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:51.004753 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5874f203-7424-4c0d-924a-0c5d5d10625f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:51.028148 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-5874f203-7424-4c0d-924a-0c5d5d10625f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:51.028148 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:51.028148 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:51.028148 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:51.028148 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:51.028148 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:51.028148 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:51.028148 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:51.028148 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:51.028148 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:51.028148 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:51.028148 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:51.028148 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:51.028148 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:51.028148 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:51.029083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:51.029083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:51.029083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:51.029083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:51.029083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:51.029083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:51.029083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:51.029083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:51.029083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:51.029083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:51.029083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:51.029083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:51.029083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:51.029083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:51.029083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:51.029083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:51.029083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:51.030855 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:51.030855 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:51.030855 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:51.030855 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:51.030855 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 330/672] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:51.031614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874f203-7424-4c0d-924a-0c5d5d10625f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:51.050377 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae1d993-2d18-4b9e-bd77-51ec27f3dd1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:51.052457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae1d993-2d18-4b9e-bd77-51ec27f3dd1f None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:51.157538 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874f203-7424-4c0d-924a-0c5d5d10625f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:51.179794 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 343/673] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:51.180669 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1d993-2d18-4b9e-bd77-51ec27f3dd1f None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:51.213749 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae6248-7f97-418b-945a-d42afb26197d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:51.215312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae6248-7f97-418b-945a-d42afb26197d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:51.215796 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae6248-7f97-418b-945a-d42afb26197d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:51.216268 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae6248-7f97-418b-945a-d42afb26197d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:51.216859 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:51.230962 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae6248-7f97-418b-945a-d42afb26197d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:51.235229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae6248-7f97-418b-945a-d42afb26197d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:51.266177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-71ae6248-7f97-418b-945a-d42afb26197d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:51.266177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:51.266177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:51.266177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:51.266177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:51.266177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:51.266177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:51.266177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:51.266177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:51.266177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:51.266177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:51.266177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:51.266177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:58:51.266177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:58:51.266177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:58:51.268709 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:58:51.268709 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:51.268709 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:51.268709 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:51.268709 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:51.268709 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:51.268709 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:51.268709 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:51.268709 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:51.268709 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:51.268709 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:51.268709 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:51.268709 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:51.268709 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:51.268709 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:51.268709 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:51.268709 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:51.271456 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:58:51.271456 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:58:51.271456 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:58:51.271456 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:58:51.271456 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:58:51.271456 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:58:51.271456 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:51.271456 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:51.271456 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 331/674] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:51.271456 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae6248-7f97-418b-945a-d42afb26197d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:51.276883 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:51.397986 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1d993-2d18-4b9e-bd77-51ec27f3dd1f None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:51.439202 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae6248-7f97-418b-945a-d42afb26197d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:51.445233 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06eda4-d656-44ef-9860-2831b6ff9ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:51.445903 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06eda4-d656-44ef-9860-2831b6ff9ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:51.446215 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06eda4-d656-44ef-9860-2831b6ff9ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:51.446493 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06eda4-d656-44ef-9860-2831b6ff9ab1 None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:51.447217 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca06eda4-d656-44ef-9860-2831b6ff9ab1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:51.447574 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca06eda4-d656-44ef-9860-2831b6ff9ab1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:51.448956 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca06eda4-d656-44ef-9860-2831b6ff9ab1 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:51.462197 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 344/675] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:51.463065 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06eda4-d656-44ef-9860-2831b6ff9ab1 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:51.482486 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c27045-b29c-4fbb-b768-a0be46247d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:51.483734 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c27045-b29c-4fbb-b768-a0be46247d3d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:51.484318 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c27045-b29c-4fbb-b768-a0be46247d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:51.484799 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c27045-b29c-4fbb-b768-a0be46247d3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:51.497665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c27045-b29c-4fbb-b768-a0be46247d3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:51.499964 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c27045-b29c-4fbb-b768-a0be46247d3d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:51.520858 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-d6c27045-b29c-4fbb-b768-a0be46247d3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:51.520858 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:51.520858 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:51.520858 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:51.520858 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:51.520858 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:51.520858 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:51.520858 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:51.520858 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:51.520858 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:51.520858 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:51.520858 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:51.520858 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:51.520858 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:51.520858 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:51.521830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:51.521830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:51.521830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:51.521830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:51.521830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:51.521830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:51.521830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:51.521830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:51.521830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:51.521830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:51.521830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:51.521830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:51.521830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:51.521830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:51.521830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:51.521830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:51.521830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:51.522761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:51.522761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:51.522761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:51.522761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:51.530380 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 332/676] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:51.531877 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c27045-b29c-4fbb-b768-a0be46247d3d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:51.550772 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:51.574552 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06eda4-d656-44ef-9860-2831b6ff9ab1 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:51.611793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:51.623910 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab51d41-1467-4c06-85d1-1158aeacbd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:51.624875 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab51d41-1467-4c06-85d1-1158aeacbd5a None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:51.625158 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab51d41-1467-4c06-85d1-1158aeacbd5a None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:51.625990 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab51d41-1467-4c06-85d1-1158aeacbd5a None admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:51.626910 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab51d41-1467-4c06-85d1-1158aeacbd5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:51.627188 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab51d41-1467-4c06-85d1-1158aeacbd5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:51.628649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab51d41-1467-4c06-85d1-1158aeacbd5a None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:51.643970 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 345/677] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:58:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:51.644924 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab51d41-1467-4c06-85d1-1158aeacbd5a None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:51.677020 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c27045-b29c-4fbb-b768-a0be46247d3d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:51.729621 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d49a37-a16f-4c71-b0c8-bc95ba89a012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:51.730407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d49a37-a16f-4c71-b0c8-bc95ba89a012 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:51.730692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d49a37-a16f-4c71-b0c8-bc95ba89a012 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:51.730967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d49a37-a16f-4c71-b0c8-bc95ba89a012 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:51.742109 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d49a37-a16f-4c71-b0c8-bc95ba89a012 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:51.744216 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d49a37-a16f-4c71-b0c8-bc95ba89a012 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:51.760175 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab51d41-1467-4c06-85d1-1158aeacbd5a None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:51.765123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-c5d49a37-a16f-4c71-b0c8-bc95ba89a012 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 15:58:51.765123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:51.765123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:51.765123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:51.765123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:51.765123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:51.765123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:51.765123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:51.765123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:51.765123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:51.765123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:51.765123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:51.765123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:58:51.765123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:58:51.765123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:58:51.765123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:58:51.766166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:51.766166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:51.766166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:58:51.766166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:51.766166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:58:51.766166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:51.766166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:51.766166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:51.766166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:51.766166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:51.766166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:51.766166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:51.766166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:51.766166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:51.766166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:51.766166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:51.766166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:51.766166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:51.769081 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:51.769081 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:51.769081 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:58:51.769081 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:58:51.769081 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:58:51.769081 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:58:51.769081 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:58:51.769081 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:58:51.769081 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 15:58:51.769081 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:51.769081 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 333/678] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 15:58:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:51.769081 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d49a37-a16f-4c71-b0c8-bc95ba89a012 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:51.816506 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6443c8-e8c8-42aa-8fab-ac1dacbeabcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:51.817336 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6443c8-e8c8-42aa-8fab-ac1dacbeabcd None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:51.817720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6443c8-e8c8-42aa-8fab-ac1dacbeabcd None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:51.818126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6443c8-e8c8-42aa-8fab-ac1dacbeabcd None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:51.819162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6443c8-e8c8-42aa-8fab-ac1dacbeabcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:51.819611 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6443c8-e8c8-42aa-8fab-ac1dacbeabcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:51.821202 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6443c8-e8c8-42aa-8fab-ac1dacbeabcd None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:51.830512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:51.839534 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 346/679] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:51.841241 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6443c8-e8c8-42aa-8fab-ac1dacbeabcd None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:51.899295 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d49a37-a16f-4c71-b0c8-bc95ba89a012 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:51.917213 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:51.939660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369535c5-c338-4435-9ed9-5f6a8c4198d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:51.940317 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369535c5-c338-4435-9ed9-5f6a8c4198d8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:51.940596 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369535c5-c338-4435-9ed9-5f6a8c4198d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:51.940867 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369535c5-c338-4435-9ed9-5f6a8c4198d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:51.941684 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369535c5-c338-4435-9ed9-5f6a8c4198d8 None admin] RBAC: Adding query filter params (role.id=cfc67444e9944ffc87173be8c9245108, scope.project.id=b1177a85a0de4276898daf4a24ff069f, user.id=69b669a5ba4e4294baeea369949e3e6f) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:51.942016 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369535c5-c338-4435-9ed9-5f6a8c4198d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:51.943510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369535c5-c338-4435-9ed9-5f6a8c4198d8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:51.950127 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 334/680] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:58:51 2022] GET /identity/v3/role_assignments?user.id=69b669a5ba4e4294baeea369949e3e6f&scope.project.id=b1177a85a0de4276898daf4a24ff069f&role.id=cfc67444e9944ffc87173be8c9245108 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:51.950730 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369535c5-c338-4435-9ed9-5f6a8c4198d8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:51.966429 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6443c8-e8c8-42aa-8fab-ac1dacbeabcd None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:52.018938 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5d5d42-88d4-444f-a93c-353d0963c79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:52.020346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d5d42-88d4-444f-a93c-353d0963c79e None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:52.020931 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d5d42-88d4-444f-a93c-353d0963c79e None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:52.021619 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d5d42-88d4-444f-a93c-353d0963c79e None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:52.023176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5d5d42-88d4-444f-a93c-353d0963c79e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:52.023702 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5d5d42-88d4-444f-a93c-353d0963c79e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:52.025418 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5d5d42-88d4-444f-a93c-353d0963c79e None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:52.047494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 347/681] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jun 16 15:58:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:52.048983 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d96f3-0470-482a-b3fb-8fc1081d9db8 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:52.049460 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d96f3-0470-482a-b3fb-8fc1081d9db8 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:52.049622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d96f3-0470-482a-b3fb-8fc1081d9db8 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:52.055710 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 348/682] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:52.056730 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d96f3-0470-482a-b3fb-8fc1081d9db8 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:52.089992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369535c5-c338-4435-9ed9-5f6a8c4198d8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:52.121485 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:52.145906 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d66f0ea-cc89-4eb6-b3d9-a2744b7cf2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:52.146838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66f0ea-cc89-4eb6-b3d9-a2744b7cf2a0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:52.147122 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66f0ea-cc89-4eb6-b3d9-a2744b7cf2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:52.147397 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66f0ea-cc89-4eb6-b3d9-a2744b7cf2a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:52.157127 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d66f0ea-cc89-4eb6-b3d9-a2744b7cf2a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:52.159288 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d66f0ea-cc89-4eb6-b3d9-a2744b7cf2a0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:52.178023 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-0d66f0ea-cc89-4eb6-b3d9-a2744b7cf2a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:52.178023 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:52.178023 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:52.178023 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:52.178023 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:52.178023 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:52.178023 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:52.178023 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:52.178023 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:52.178023 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:52.178023 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:52.178023 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:52.178023 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:58:52.178023 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:58:52.178023 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:58:52.179279 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:58:52.179279 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:52.179279 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:52.179279 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:52.179279 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:52.179279 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:52.179279 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:52.179279 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:52.179279 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:52.179279 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:52.179279 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:52.179279 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:52.179279 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:52.179279 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:52.179279 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:52.179279 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:52.179279 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:52.180292 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:58:52.180292 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:58:52.180292 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:58:52.180292 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:58:52.180292 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:58:52.180292 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:58:52.180292 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:52.180292 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:52.180292 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 335/683] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:52.180814 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828361d-2eb6-4612-9458-89304baeedb0 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:52.180814 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828361d-2eb6-4612-9458-89304baeedb0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:52.181320 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828361d-2eb6-4612-9458-89304baeedb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:52.198654 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d96f3-0470-482a-b3fb-8fc1081d9db8 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:52.210623 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4828361d-2eb6-4612-9458-89304baeedb0 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:52.211847 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4828361d-2eb6-4612-9458-89304baeedb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:52.246342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f11a9-2930-46de-93ce-1a723ae4bc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:52.247361 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f11a9-2930-46de-93ce-1a723ae4bc9d None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:52.247599 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f11a9-2930-46de-93ce-1a723ae4bc9d None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:52.247887 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f11a9-2930-46de-93ce-1a723ae4bc9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:52.258297 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:52.293758 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 336/684] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:52.294789 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4828361d-2eb6-4612-9458-89304baeedb0 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:52.340966 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f11a9-2930-46de-93ce-1a723ae4bc9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:52.342577 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f11a9-2930-46de-93ce-1a723ae4bc9d None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:52.405666 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:52.414882 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4828361d-2eb6-4612-9458-89304baeedb0 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:52.451779 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207fea1-f4c4-4076-8a67-fa58149e9888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:52.452700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207fea1-f4c4-4076-8a67-fa58149e9888 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:52.453048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207fea1-f4c4-4076-8a67-fa58149e9888 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:52.453329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207fea1-f4c4-4076-8a67-fa58149e9888 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:52.454215 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5207fea1-f4c4-4076-8a67-fa58149e9888 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:52.454528 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5207fea1-f4c4-4076-8a67-fa58149e9888 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:52.456111 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5207fea1-f4c4-4076-8a67-fa58149e9888 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:52.458188 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 349/685] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:52.459084 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f11a9-2930-46de-93ce-1a723ae4bc9d None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:52.474179 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 337/686] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:58:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:52.475087 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207fea1-f4c4-4076-8a67-fa58149e9888 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:52.515057 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:52.567378 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f11a9-2930-46de-93ce-1a723ae4bc9d None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:52.587617 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207fea1-f4c4-4076-8a67-fa58149e9888 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:52.604399 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b685d2-f949-4d12-8b5b-c60c73eff13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:52.605479 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b685d2-f949-4d12-8b5b-c60c73eff13b None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:52.605763 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b685d2-f949-4d12-8b5b-c60c73eff13b None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:52.606076 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b685d2-f949-4d12-8b5b-c60c73eff13b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:52.622632 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b685d2-f949-4d12-8b5b-c60c73eff13b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:52.627309 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1506df-0267-4172-a8ec-0ebf9d018826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:52.628099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1506df-0267-4172-a8ec-0ebf9d018826 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:52.628377 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1506df-0267-4172-a8ec-0ebf9d018826 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:52.628649 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1506df-0267-4172-a8ec-0ebf9d018826 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:52.632021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-c7b685d2-f949-4d12-8b5b-c60c73eff13b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:52.632021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:52.632021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:52.632021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:52.632021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:52.632021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:52.632021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:52.632021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:52.632021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:52.632021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:52.632021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:52.632021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:52.632021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:52.632021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:52.632021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:52.632021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:52.634455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:52.634455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:52.634455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:52.634455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:52.634455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:52.634455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:52.634455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:52.634455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:52.634455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:52.634455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:52.634455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:52.634455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:52.634455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:52.634455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:52.634455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:52.634455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:52.636404 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:52.636404 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:52.636404 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:52.636404 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:52.636404 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:52.636404 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:52.636404 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:52.636404 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:52.636404 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:52.636404 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:52.636404 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:52.636404 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:52.636404 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 350/687] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 15:58:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:52.636404 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b685d2-f949-4d12-8b5b-c60c73eff13b None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:52.645867 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1506df-0267-4172-a8ec-0ebf9d018826 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:52.647377 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1506df-0267-4172-a8ec-0ebf9d018826 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:52.666473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-be1506df-0267-4172-a8ec-0ebf9d018826 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:52.666473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:52.666473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:52.666473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:52.666473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:52.666473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:52.666473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:52.666473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:52.666473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:52.666473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:52.666473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:52.666473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:52.666473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:52.666473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:52.666473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:52.668154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:52.668154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:52.668154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:52.668154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:52.668154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:52.668154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:52.668154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:52.668154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:52.668154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:52.668154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:52.668154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:52.668154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:52.668154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:52.668154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:52.668154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:52.668154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:52.668154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:52.669121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:52.669121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:52.669121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:52.669121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:52.669121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 338/688] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:52.669121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1506df-0267-4172-a8ec-0ebf9d018826 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:52.690333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:52.770046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:52.782918 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b685d2-f949-4d12-8b5b-c60c73eff13b None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:52.798642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1506df-0267-4172-a8ec-0ebf9d018826 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:52.830259 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d445f9f-1bcd-4087-a26b-0b476c0d5afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:52.831451 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d445f9f-1bcd-4087-a26b-0b476c0d5afd None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:52.831899 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d445f9f-1bcd-4087-a26b-0b476c0d5afd None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:52.832312 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d445f9f-1bcd-4087-a26b-0b476c0d5afd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:52.833309 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d445f9f-1bcd-4087-a26b-0b476c0d5afd None admin] RBAC: Adding query filter params (role.id=cfc67444e9944ffc87173be8c9245108, scope.project.id=b1177a85a0de4276898daf4a24ff069f, user.id=5a84f8aed50c4b3e8e5b33c5b587b26e) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:52.833792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d445f9f-1bcd-4087-a26b-0b476c0d5afd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:52.835302 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e808f2-868f-4ff1-8920-7fedd47db11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:52.835621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d445f9f-1bcd-4087-a26b-0b476c0d5afd None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:52.836185 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e808f2-868f-4ff1-8920-7fedd47db11e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:52.836568 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e808f2-868f-4ff1-8920-7fedd47db11e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:52.836943 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e808f2-868f-4ff1-8920-7fedd47db11e None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:52.837792 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e808f2-868f-4ff1-8920-7fedd47db11e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:52.838228 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e808f2-868f-4ff1-8920-7fedd47db11e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:52.840052 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e808f2-868f-4ff1-8920-7fedd47db11e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:52.845014 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 351/689] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:58:52 2022] GET /identity/v3/role_assignments?user.id=5a84f8aed50c4b3e8e5b33c5b587b26e&scope.project.id=b1177a85a0de4276898daf4a24ff069f&role.id=cfc67444e9944ffc87173be8c9245108 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:52.846737 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d445f9f-1bcd-4087-a26b-0b476c0d5afd None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:52.850327 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 339/690] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:52.855818 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e808f2-868f-4ff1-8920-7fedd47db11e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:52.973814 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d445f9f-1bcd-4087-a26b-0b476c0d5afd None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:52.978164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:53.018169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f198e63d-6752-4c77-91d2-51bfcd509bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:53.019298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f198e63d-6752-4c77-91d2-51bfcd509bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:53.019759 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f198e63d-6752-4c77-91d2-51bfcd509bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:53.020681 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f198e63d-6752-4c77-91d2-51bfcd509bb9 None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:53.021892 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f198e63d-6752-4c77-91d2-51bfcd509bb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:53.022654 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f198e63d-6752-4c77-91d2-51bfcd509bb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:53.024502 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f198e63d-6752-4c77-91d2-51bfcd509bb9 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:53.039104 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 352/691] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:53.048851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f198e63d-6752-4c77-91d2-51bfcd509bb9 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:53.109715 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e808f2-868f-4ff1-8920-7fedd47db11e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:53.149023 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925f4ab-7111-47b2-84c0-5ebf01be9f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:53.150751 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925f4ab-7111-47b2-84c0-5ebf01be9f58 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:53.151261 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925f4ab-7111-47b2-84c0-5ebf01be9f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:53.152038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925f4ab-7111-47b2-84c0-5ebf01be9f58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:53.166508 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925f4ab-7111-47b2-84c0-5ebf01be9f58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:53.168863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925f4ab-7111-47b2-84c0-5ebf01be9f58 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:53.175718 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:53.195890 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f198e63d-6752-4c77-91d2-51bfcd509bb9 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:53.199791 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-6925f4ab-7111-47b2-84c0-5ebf01be9f58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:53.199791 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:53.199791 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:53.199791 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:53.199791 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:53.199791 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:53.199791 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:53.199791 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:53.199791 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:53.199791 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:53.199791 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:53.199791 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:53.199791 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:53.199791 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:53.199791 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:53.200764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:53.200764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:53.200764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:53.200764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:53.200764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:53.200764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:53.200764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:53.200764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:53.200764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:53.200764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:53.200764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:53.200764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:53.200764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:53.200764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:53.200764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:53.200764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:53.200764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:53.201768 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:53.201768 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:53.201768 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:53.201768 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:53.204275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 340/692] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:53.213619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925f4ab-7111-47b2-84c0-5ebf01be9f58 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:53.249792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f59e7c3-dc44-45ec-9862-fb63cc4c8d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:53.250649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59e7c3-dc44-45ec-9862-fb63cc4c8d6b None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:53.250994 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59e7c3-dc44-45ec-9862-fb63cc4c8d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:53.251582 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59e7c3-dc44-45ec-9862-fb63cc4c8d6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:53.259137 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f59e7c3-dc44-45ec-9862-fb63cc4c8d6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:53.262019 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f59e7c3-dc44-45ec-9862-fb63cc4c8d6b None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:53.265322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:53.279617 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-2f59e7c3-dc44-45ec-9862-fb63cc4c8d6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:53.279617 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:53.279617 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:53.279617 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:53.279617 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:53.279617 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:53.279617 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:53.279617 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:53.279617 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:53.279617 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:53.279617 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:53.279617 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:53.279617 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:58:53.279617 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:58:53.279617 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:58:53.281896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:58:53.281896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:53.281896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:53.281896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:53.281896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:53.281896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:53.281896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:53.281896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:53.281896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:53.281896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:53.281896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:53.281896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:53.281896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:53.281896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:53.281896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:53.281896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:53.281896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:53.284076 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:58:53.284076 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:58:53.284076 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:58:53.284076 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:58:53.284076 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:58:53.284076 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:58:53.284076 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:53.284076 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:53.284076 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 353/693] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:53.284076 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae8558-8024-441c-bf08-2d59464c3d25 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:53.284076 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae8558-8024-441c-bf08-2d59464c3d25 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:53.285644 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae8558-8024-441c-bf08-2d59464c3d25 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:53.289238 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 354/694] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:53.291470 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae8558-8024-441c-bf08-2d59464c3d25 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:53.341378 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925f4ab-7111-47b2-84c0-5ebf01be9f58 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:53.380720 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d0d67-1807-4946-928f-ce64bb6038d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:53.381293 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d0d67-1807-4946-928f-ce64bb6038d9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:53.381576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d0d67-1807-4946-928f-ce64bb6038d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:53.381851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d0d67-1807-4946-928f-ce64bb6038d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:53.382616 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d0d67-1807-4946-928f-ce64bb6038d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:53.382954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d0d67-1807-4946-928f-ce64bb6038d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:53.384748 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d0d67-1807-4946-928f-ce64bb6038d9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:53.400093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 341/695] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:53.401395 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8433419-a0cf-4a6c-8beb-d46ac18d9a2e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:53.401688 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8433419-a0cf-4a6c-8beb-d46ac18d9a2e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:53.406518 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8433419-a0cf-4a6c-8beb-d46ac18d9a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:53.425939 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f8433419-a0cf-4a6c-8beb-d46ac18d9a2e None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:53.427596 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f8433419-a0cf-4a6c-8beb-d46ac18d9a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:53.441214 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae8558-8024-441c-bf08-2d59464c3d25 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:53.487384 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:53.491447 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc5c98-680a-4648-8afa-55153d65ee97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:53.492338 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc5c98-680a-4648-8afa-55153d65ee97 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:53.492818 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc5c98-680a-4648-8afa-55153d65ee97 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:53.494508 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc5c98-680a-4648-8afa-55153d65ee97 None admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:53.495673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebc5c98-680a-4648-8afa-55153d65ee97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:53.496318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebc5c98-680a-4648-8afa-55153d65ee97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:53.498755 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebc5c98-680a-4648-8afa-55153d65ee97 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:53.517955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 342/696] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:53.517955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8433419-a0cf-4a6c-8beb-d46ac18d9a2e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:53.527907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 355/697] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:58:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:53.535219 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc5c98-680a-4648-8afa-55153d65ee97 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:53.600777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:53.647511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc5c98-680a-4648-8afa-55153d65ee97 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:53.653940 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8433419-a0cf-4a6c-8beb-d46ac18d9a2e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:53.683834 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1caf7-f53f-4dd5-b9f4-34b59da25f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:53.684498 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1caf7-f53f-4dd5-b9f4-34b59da25f52 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:53.684782 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1caf7-f53f-4dd5-b9f4-34b59da25f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:53.685059 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1caf7-f53f-4dd5-b9f4-34b59da25f52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:53.686803 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba1caf7-f53f-4dd5-b9f4-34b59da25f52 None admin] RBAC: Adding query filter params (role.id=cfc67444e9944ffc87173be8c9245108, scope.project.id=b1177a85a0de4276898daf4a24ff069f, user.id=0082e37ce54f49f7837601349508c89f) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:53.687381 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba1caf7-f53f-4dd5-b9f4-34b59da25f52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:53.689440 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba1caf7-f53f-4dd5-b9f4-34b59da25f52 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:53.695272 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f42a1-a440-46ff-bc4d-78b7229af489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:53.696547 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f42a1-a440-46ff-bc4d-78b7229af489 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:53.697059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f42a1-a440-46ff-bc4d-78b7229af489 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:53.697572 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f42a1-a440-46ff-bc4d-78b7229af489 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:53.702206 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 356/698] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:58:53 2022] GET /identity/v3/role_assignments?user.id=0082e37ce54f49f7837601349508c89f&scope.project.id=b1177a85a0de4276898daf4a24ff069f&role.id=cfc67444e9944ffc87173be8c9245108 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:53.703753 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1caf7-f53f-4dd5-b9f4-34b59da25f52 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:53.716631 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f42a1-a440-46ff-bc4d-78b7229af489 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:53.723297 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f42a1-a440-46ff-bc4d-78b7229af489 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:53.760933 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-028f42a1-a440-46ff-bc4d-78b7229af489 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 15:58:53.760933 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:53.760933 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:53.760933 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:53.760933 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:53.760933 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:53.760933 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:53.760933 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:53.760933 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:53.760933 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:53.760933 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:53.760933 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:53.760933 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:58:53.760933 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:58:53.760933 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:58:53.760933 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:58:53.762034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:53.762034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:53.762034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:58:53.762034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:53.762034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:58:53.762034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:53.762034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:53.762034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:53.762034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:53.762034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:53.762034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:53.762034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:53.762034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:53.762034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:53.762034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:53.762034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:53.762034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:53.762034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:53.763054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:53.763054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:53.763054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:58:53.763054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:58:53.763054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:58:53.763054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:58:53.763054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:58:53.763054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:58:53.763054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 15:58:53.763054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:53.769085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 343/699] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jun 16 15:58:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:53.773818 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f42a1-a440-46ff-bc4d-78b7229af489 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:53.885463 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:53.919030 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1caf7-f53f-4dd5-b9f4-34b59da25f52 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:53.949305 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:53.970517 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1131f73e-7044-44b6-9a8a-716ef89e98d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:53.971330 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131f73e-7044-44b6-9a8a-716ef89e98d6 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:53.971611 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131f73e-7044-44b6-9a8a-716ef89e98d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:53.971882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131f73e-7044-44b6-9a8a-716ef89e98d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:53.987579 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1131f73e-7044-44b6-9a8a-716ef89e98d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:53.989057 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1131f73e-7044-44b6-9a8a-716ef89e98d6 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:53.990041 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f42a1-a440-46ff-bc4d-78b7229af489 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:54.013578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-1131f73e-7044-44b6-9a8a-716ef89e98d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:54.013578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:54.013578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:54.013578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:54.013578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:54.013578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:54.013578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:54.013578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:54.013578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:54.013578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:54.013578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:54.013578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:54.013578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:54.013578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:54.013578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:54.014649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:54.014649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:54.014649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:54.014649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:54.014649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:54.014649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:54.014649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:54.014649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:54.014649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:54.014649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:54.014649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:54.014649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:54.014649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:54.014649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:54.014649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:54.014649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:54.014649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:54.015547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:54.015547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:54.015547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:54.015547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:54.015547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 357/700] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:54.021245 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1131f73e-7044-44b6-9a8a-716ef89e98d6 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:54.034561 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58c77e-dea3-42b5-9c08-bfb53eb31fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:54.035650 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58c77e-dea3-42b5-9c08-bfb53eb31fec None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:54.036223 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58c77e-dea3-42b5-9c08-bfb53eb31fec None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:54.036717 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58c77e-dea3-42b5-9c08-bfb53eb31fec None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:54.038002 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58c77e-dea3-42b5-9c08-bfb53eb31fec None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:54.038552 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58c77e-dea3-42b5-9c08-bfb53eb31fec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:54.040443 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58c77e-dea3-42b5-9c08-bfb53eb31fec None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:54.064209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 344/701] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jun 16 15:58:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:54.071747 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58c77e-dea3-42b5-9c08-bfb53eb31fec None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:54.134636 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1131f73e-7044-44b6-9a8a-716ef89e98d6 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:54.172737 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac0736-00ac-40d0-ac5b-bb15bfea4a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:54.173329 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac0736-00ac-40d0-ac5b-bb15bfea4a55 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:54.173613 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac0736-00ac-40d0-ac5b-bb15bfea4a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:54.173895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac0736-00ac-40d0-ac5b-bb15bfea4a55 None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:54.174652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfac0736-00ac-40d0-ac5b-bb15bfea4a55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:54.174960 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfac0736-00ac-40d0-ac5b-bb15bfea4a55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:54.176409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfac0736-00ac-40d0-ac5b-bb15bfea4a55 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:54.189072 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58c77e-dea3-42b5-9c08-bfb53eb31fec None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:54.189874 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 358/702] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:54.192263 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:54.195484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac0736-00ac-40d0-ac5b-bb15bfea4a55 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:54.230599 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0eb62-4e0e-45d2-b800-20d3207b2738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:54.231730 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0eb62-4e0e-45d2-b800-20d3207b2738 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:54.232229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0eb62-4e0e-45d2-b800-20d3207b2738 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:54.233269 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0eb62-4e0e-45d2-b800-20d3207b2738 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:54.234778 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:54.317529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac0736-00ac-40d0-ac5b-bb15bfea4a55 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:54.319680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c0eb62-4e0e-45d2-b800-20d3207b2738 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:54.321587 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c0eb62-4e0e-45d2-b800-20d3207b2738 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:54.356887 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6cc7a-3264-4494-bfc3-149ec521f11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:54.357658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6cc7a-3264-4494-bfc3-149ec521f11b None admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:54.357999 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6cc7a-3264-4494-bfc3-149ec521f11b None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:54.358472 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6cc7a-3264-4494-bfc3-149ec521f11b None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:54.359316 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de6cc7a-3264-4494-bfc3-149ec521f11b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:54.361439 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de6cc7a-3264-4494-bfc3-149ec521f11b None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:54.384378 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 359/703] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 15:58:54 2022] POST /identity/v3/users => generated 254 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:58:54.385833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40540d-e396-48bd-8bb3-6f84631d2870 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:54.390444 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40540d-e396-48bd-8bb3-6f84631d2870 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:54.390864 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40540d-e396-48bd-8bb3-6f84631d2870 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:54.392815 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 360/704] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:54.404722 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459131b9-5437-4412-b5cf-b02078f04fee None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:54.405032 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459131b9-5437-4412-b5cf-b02078f04fee None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:54.405280 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459131b9-5437-4412-b5cf-b02078f04fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:54.440686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-459131b9-5437-4412-b5cf-b02078f04fee None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:54.441955 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-459131b9-5437-4412-b5cf-b02078f04fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:54.448397 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 345/705] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:54.458037 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0eb62-4e0e-45d2-b800-20d3207b2738 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:54.471572 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:54.606088 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 361/706] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:58:54.634934 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459131b9-5437-4412-b5cf-b02078f04fee None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:54.648995 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0eb62-4e0e-45d2-b800-20d3207b2738 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:54.663918 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:54.699749 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226defe2-90c1-41dc-b651-0a368dda992e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:54.700593 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226defe2-90c1-41dc-b651-0a368dda992e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:54.700883 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226defe2-90c1-41dc-b651-0a368dda992e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:54.701232 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226defe2-90c1-41dc-b651-0a368dda992e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:54.716734 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226defe2-90c1-41dc-b651-0a368dda992e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:54.718938 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226defe2-90c1-41dc-b651-0a368dda992e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:54.741579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-226defe2-90c1-41dc-b651-0a368dda992e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:54.741579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:54.741579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:54.741579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:54.741579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:54.741579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:54.741579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:54.741579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:54.741579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:54.741579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:54.741579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:54.741579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:54.741579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:54.741579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:54.741579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:54.742614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:54.742614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:54.742614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:54.742614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:54.742614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:54.742614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:54.742614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:54.742614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:54.742614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:54.742614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:54.742614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:54.742614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:54.742614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:54.742614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:54.742614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:54.742614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:54.742614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:54.743542 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:54.743542 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:54.743542 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:54.743542 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:54.743542 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 346/707] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:54.749859 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226defe2-90c1-41dc-b651-0a368dda992e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:54.762274 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459131b9-5437-4412-b5cf-b02078f04fee None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:54.817300 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39adcd-2e0c-4119-87d2-3ce510e82631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:54.818783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39adcd-2e0c-4119-87d2-3ce510e82631 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:54.819743 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39adcd-2e0c-4119-87d2-3ce510e82631 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:54.820254 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39adcd-2e0c-4119-87d2-3ce510e82631 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:54.821370 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da39adcd-2e0c-4119-87d2-3ce510e82631 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:54.823396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da39adcd-2e0c-4119-87d2-3ce510e82631 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:54.834446 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:54.840634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-da39adcd-2e0c-4119-87d2-3ce510e82631 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 15:58:54.840634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:54.840634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 15:58:54.840634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 15:58:54.840634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 15:58:54.840634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 15:58:54.840634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 15:58:54.840634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:58:54.840634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 15:58:54.840634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:54.840634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:58:54.840634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:54.840634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:54.840634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:54.840634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:54.840634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:54.841693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:54.841693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:54.841693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:54.841693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:54.841693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:54.841693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:54.841693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:54.841693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 15:58:54.841693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 15:58:54.841693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 15:58:54.841693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 15:58:54.841693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:54.841693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:54.841693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:54.841693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:54.841693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:54.843014 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:54.843014 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:54.843014 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:54.843014 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:54.843014 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:54.843014 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:54.843014 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:54.843014 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:54.843014 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:54.843014 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:54.843014 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:54.843014 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 15:58:54.843014 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:58:54.843014 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 15:58:54.843014 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:54.846294 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 362/708] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jun 16 15:58:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:54.848241 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39adcd-2e0c-4119-87d2-3ce510e82631 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:54.902625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226defe2-90c1-41dc-b651-0a368dda992e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:54.943010 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3ab3d-3625-46c2-bdc9-744107585296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:54.943889 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3ab3d-3625-46c2-bdc9-744107585296 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:54.944183 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3ab3d-3625-46c2-bdc9-744107585296 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:54.944466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3ab3d-3625-46c2-bdc9-744107585296 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:54.945200 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc3ab3d-3625-46c2-bdc9-744107585296 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:54.945510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc3ab3d-3625-46c2-bdc9-744107585296 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:54.947079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc3ab3d-3625-46c2-bdc9-744107585296 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:54.961281 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 347/709] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:54.962520 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:54.967277 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3ab3d-3625-46c2-bdc9-744107585296 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:54.983184 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39adcd-2e0c-4119-87d2-3ce510e82631 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:55.032182 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a615520-013d-4ec4-89f9-5c32cb2f5e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:55.033502 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a615520-013d-4ec4-89f9-5c32cb2f5e63 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:55.034638 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a615520-013d-4ec4-89f9-5c32cb2f5e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:55.035375 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a615520-013d-4ec4-89f9-5c32cb2f5e63 None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:55.036564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a615520-013d-4ec4-89f9-5c32cb2f5e63 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:55.037476 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a615520-013d-4ec4-89f9-5c32cb2f5e63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:55.039437 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a615520-013d-4ec4-89f9-5c32cb2f5e63 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:55.046310 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 363/710] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jun 16 15:58:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:55.055188 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a615520-013d-4ec4-89f9-5c32cb2f5e63 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:55.119736 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3ab3d-3625-46c2-bdc9-744107585296 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:55.162310 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:55.178319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d167c84-f040-4017-a293-995f68b35c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:55.179756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d167c84-f040-4017-a293-995f68b35c87 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:55.180042 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d167c84-f040-4017-a293-995f68b35c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:55.180314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d167c84-f040-4017-a293-995f68b35c87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:55.194631 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d167c84-f040-4017-a293-995f68b35c87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:55.196096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d167c84-f040-4017-a293-995f68b35c87 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:55.221784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-7d167c84-f040-4017-a293-995f68b35c87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:55.221784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:55.221784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:55.221784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:55.221784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:55.221784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:55.221784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:55.221784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:55.221784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:55.221784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:55.221784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:55.221784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:55.221784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:58:55.221784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:58:55.221784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:58:55.222731 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:58:55.222731 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:55.222731 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:55.222731 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:55.222731 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:55.222731 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:55.222731 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:55.222731 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:55.222731 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:55.222731 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:55.222731 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:55.222731 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:55.222731 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:55.222731 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:55.222731 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:55.222731 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:55.222731 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:55.223623 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:58:55.223623 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:58:55.223623 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:58:55.223623 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:58:55.223623 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:58:55.223623 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:58:55.223623 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:55.223623 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:55.223623 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 348/711] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:55.228766 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d167c84-f040-4017-a293-995f68b35c87 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:55.274229 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a615520-013d-4ec4-89f9-5c32cb2f5e63 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:55.337549 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01528d9e-d5bb-4aa8-9200-a1cde5e95a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:55.338247 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01528d9e-d5bb-4aa8-9200-a1cde5e95a32 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:55.338528 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01528d9e-d5bb-4aa8-9200-a1cde5e95a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:55.338803 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01528d9e-d5bb-4aa8-9200-a1cde5e95a32 None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:55.339512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01528d9e-d5bb-4aa8-9200-a1cde5e95a32 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:55.339817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01528d9e-d5bb-4aa8-9200-a1cde5e95a32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:55.341198 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01528d9e-d5bb-4aa8-9200-a1cde5e95a32 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:55.349751 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 364/712] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jun 16 15:58:55 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:55.432757 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d167c84-f040-4017-a293-995f68b35c87 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:55.492374 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bad76d-ec50-4dc7-8f05-0f70257d9a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:55.493027 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bad76d-ec50-4dc7-8f05-0f70257d9a69 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:55.493316 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bad76d-ec50-4dc7-8f05-0f70257d9a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:55.493597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bad76d-ec50-4dc7-8f05-0f70257d9a69 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:55.494437 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bad76d-ec50-4dc7-8f05-0f70257d9a69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:55.494752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bad76d-ec50-4dc7-8f05-0f70257d9a69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:55.496236 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bad76d-ec50-4dc7-8f05-0f70257d9a69 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:55.515857 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 349/713] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:58:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:55.521841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01528d9e-d5bb-4aa8-9200-a1cde5e95a32 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:55.546151 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:55.660868 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01528d9e-d5bb-4aa8-9200-a1cde5e95a32 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:55.674783 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85464df-9ff3-40cf-89bb-39548c927d3a None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:55.675082 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85464df-9ff3-40cf-89bb-39548c927d3a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:55.675356 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85464df-9ff3-40cf-89bb-39548c927d3a None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:55.678017 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 350/714] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:55.683034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250c8ac-8a4a-45c9-b658-ad4a5aee1183 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:55.683328 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250c8ac-8a4a-45c9-b658-ad4a5aee1183 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:55.683608 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250c8ac-8a4a-45c9-b658-ad4a5aee1183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:55.706995 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-5250c8ac-8a4a-45c9-b658-ad4a5aee1183 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:55.708197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5250c8ac-8a4a-45c9-b658-ad4a5aee1183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:55.710787 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904b7f4-a6f2-48f5-a347-d2bf75de75d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:55.711458 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904b7f4-a6f2-48f5-a347-d2bf75de75d3 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:55.711849 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904b7f4-a6f2-48f5-a347-d2bf75de75d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:55.712127 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904b7f4-a6f2-48f5-a347-d2bf75de75d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:55.712968 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904b7f4-a6f2-48f5-a347-d2bf75de75d3 None admin] RBAC: Adding query filter params (role.id=bbbc77ec645d4ff5a8bf29ce0377f578, scope.project.id=b1177a85a0de4276898daf4a24ff069f, user.id=c3645b1a0be34c01ac87c734fcd85ff6) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:55.713289 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904b7f4-a6f2-48f5-a347-d2bf75de75d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:55.714760 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904b7f4-a6f2-48f5-a347-d2bf75de75d3 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:55.726106 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 365/715] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:58:55 2022] GET /identity/v3/role_assignments?user.id=c3645b1a0be34c01ac87c734fcd85ff6&scope.project.id=b1177a85a0de4276898daf4a24ff069f&role.id=bbbc77ec645d4ff5a8bf29ce0377f578 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:55.762749 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:55.801656 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 351/716] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:55.838639 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904b7f4-a6f2-48f5-a347-d2bf75de75d3 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:55.967448 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:55.993814 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904b7f4-a6f2-48f5-a347-d2bf75de75d3 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:56.032712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88acc2da-d4a5-4a19-b2de-6f05fa94e40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:56.033471 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88acc2da-d4a5-4a19-b2de-6f05fa94e40f None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:56.033754 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88acc2da-d4a5-4a19-b2de-6f05fa94e40f None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:56.034069 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88acc2da-d4a5-4a19-b2de-6f05fa94e40f None admin] PATH_INFO: `/v3/services/network` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:56.034755 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88acc2da-d4a5-4a19-b2de-6f05fa94e40f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:56.036222 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88acc2da-d4a5-4a19-b2de-6f05fa94e40f None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:56.049652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-88acc2da-d4a5-4a19-b2de-6f05fa94e40f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 15:58:56.049652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:56.049652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 15:58:56.049652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 15:58:56.049652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 15:58:56.049652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 15:58:56.049652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 15:58:56.049652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:58:56.049652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 15:58:56.049652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:56.049652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:58:56.049652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:56.049652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:56.049652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:56.049652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:56.049652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:56.049652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:56.052048 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:56.052048 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:56.052048 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:56.052048 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:56.052048 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:56.052048 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:56.052048 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 15:58:56.052048 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 15:58:56.052048 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 15:58:56.052048 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 15:58:56.052048 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:56.052048 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:56.052048 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:56.052048 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:56.052048 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:56.052048 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:56.054491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:56.054491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:56.054491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:56.054491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:56.054491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:56.054491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:56.054491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:56.054491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:56.054491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:56.054491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:56.054491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 15:58:56.054491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:58:56.054491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 15:58:56.054491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:56.061950 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 366/717] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:55 2022] GET /identity/v3/services/network => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:56.062255 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250c8ac-8a4a-45c9-b658-ad4a5aee1183 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:56.172179 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250c8ac-8a4a-45c9-b658-ad4a5aee1183 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:56.223049 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35758157-9358-4b75-b14e-7c92ea2ca556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:56.223735 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35758157-9358-4b75-b14e-7c92ea2ca556 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:56.224022 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35758157-9358-4b75-b14e-7c92ea2ca556 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:56.224301 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35758157-9358-4b75-b14e-7c92ea2ca556 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:56.225020 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35758157-9358-4b75-b14e-7c92ea2ca556 None admin] RBAC: Adding query filter params (name=network) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:56.225329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35758157-9358-4b75-b14e-7c92ea2ca556 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:56.226725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35758157-9358-4b75-b14e-7c92ea2ca556 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:56.237997 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 352/718] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jun 16 15:58:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:56.243708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88acc2da-d4a5-4a19-b2de-6f05fa94e40f None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:56.386960 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88acc2da-d4a5-4a19-b2de-6f05fa94e40f None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:56.439765 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c05d1a-fb38-465f-902e-178a46a03ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:56.440463 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c05d1a-fb38-465f-902e-178a46a03ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:56.440746 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c05d1a-fb38-465f-902e-178a46a03ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:56.441022 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c05d1a-fb38-465f-902e-178a46a03ae3 None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:56.441746 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c05d1a-fb38-465f-902e-178a46a03ae3 None admin] RBAC: Adding query filter params (type=network) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:56.442087 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c05d1a-fb38-465f-902e-178a46a03ae3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:56.443478 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c05d1a-fb38-465f-902e-178a46a03ae3 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:56.454200 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 367/719] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jun 16 15:58:56 2022] GET /identity/v3/services?type=network => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:56.625620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d1b83-0308-4300-b92d-41042f6e198b None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:56.626312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d1b83-0308-4300-b92d-41042f6e198b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:56.626802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d1b83-0308-4300-b92d-41042f6e198b None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:56.628756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 353/720] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:56.635442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b859f32-a870-46ee-bea5-20cab00501b1 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:56.636401 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b859f32-a870-46ee-bea5-20cab00501b1 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:56.636928 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b859f32-a870-46ee-bea5-20cab00501b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:56.650161 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:56.663293 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-1b859f32-a870-46ee-bea5-20cab00501b1 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:56.664597 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-1b859f32-a870-46ee-bea5-20cab00501b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:56.761462 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 368/721] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:56.787050 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d1b83-0308-4300-b92d-41042f6e198b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:56.822588 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:56.903590 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d1b83-0308-4300-b92d-41042f6e198b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:56.941884 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd64504-863b-429c-b0ea-744f924e9f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:56.942720 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd64504-863b-429c-b0ea-744f924e9f78 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:56.943009 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd64504-863b-429c-b0ea-744f924e9f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:56.943291 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd64504-863b-429c-b0ea-744f924e9f78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:56.954918 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd64504-863b-429c-b0ea-744f924e9f78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:56.962831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-1cd64504-863b-429c-b0ea-744f924e9f78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:56.962831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:56.962831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:56.962831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:56.962831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:56.962831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:56.962831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:56.962831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:56.962831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:56.962831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:56.962831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:56.962831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:56.962831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:56.962831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:56.962831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:56.962831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:56.963756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:56.963756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:56.963756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:56.963756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:56.963756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:56.963756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:56.963756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:56.963756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:56.963756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:56.963756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:56.963756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:56.963756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:56.963756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:56.963756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:56.963756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:56.963756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:56.964599 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:56.964599 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:56.964599 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:56.964599 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:56.964599 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:56.964599 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:56.964599 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:56.964599 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:56.964599 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:56.964599 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:56.964599 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:56.964599 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:56.964599 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 354/722] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 15:58:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:56.970564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b859f32-a870-46ee-bea5-20cab00501b1 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:57.103219 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b859f32-a870-46ee-bea5-20cab00501b1 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:57.145424 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dcd044-e69e-4070-9d7a-3c9aa8687a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:57.146584 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dcd044-e69e-4070-9d7a-3c9aa8687a4d None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:57.147090 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dcd044-e69e-4070-9d7a-3c9aa8687a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:57.147569 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dcd044-e69e-4070-9d7a-3c9aa8687a4d None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:57.148522 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dcd044-e69e-4070-9d7a-3c9aa8687a4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:57.149046 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dcd044-e69e-4070-9d7a-3c9aa8687a4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:57.150699 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dcd044-e69e-4070-9d7a-3c9aa8687a4d None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:57.153502 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:57.163529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 369/723] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:57.168802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd64504-863b-429c-b0ea-744f924e9f78 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:57.300575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:57.316042 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd64504-863b-429c-b0ea-744f924e9f78 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:57.374165 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3923e07-1bde-4770-aada-2ef73822ce07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:57.374989 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3923e07-1bde-4770-aada-2ef73822ce07 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:57.375274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3923e07-1bde-4770-aada-2ef73822ce07 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:57.375549 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3923e07-1bde-4770-aada-2ef73822ce07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:57.396329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3923e07-1bde-4770-aada-2ef73822ce07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:57.397823 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3923e07-1bde-4770-aada-2ef73822ce07 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:57.422774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-d3923e07-1bde-4770-aada-2ef73822ce07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:57.422774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:57.422774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:57.422774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:57.422774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:57.422774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:57.422774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:57.422774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:57.422774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:57.422774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:57.422774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:57.422774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:57.422774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:57.422774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:57.422774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:57.423730 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:57.423730 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:57.423730 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:57.423730 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:57.423730 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:57.423730 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:57.423730 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:57.423730 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:57.423730 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:57.423730 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:57.423730 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:57.423730 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:57.423730 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:57.423730 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:57.423730 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:57.423730 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:57.423730 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:57.425269 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:57.425269 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:57.425269 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:57.425269 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:57.425269 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 355/724] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:57.429896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dcd044-e69e-4070-9d7a-3c9aa8687a4d None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:57.462804 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3154d-1eac-4d17-a9d8-49d8b8165ef7 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:57.463332 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3154d-1eac-4d17-a9d8-49d8b8165ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:57.463765 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3154d-1eac-4d17-a9d8-49d8b8165ef7 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:57.465575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 356/725] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:58:57.476214 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f26a0-36c1-4a07-88a2-8ee4f0f58a8f None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:57.476723 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f26a0-36c1-4a07-88a2-8ee4f0f58a8f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:57.477287 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f26a0-36c1-4a07-88a2-8ee4f0f58a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:57.516163 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-7f9f26a0-36c1-4a07-88a2-8ee4f0f58a8f None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:57.517667 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7f9f26a0-36c1-4a07-88a2-8ee4f0f58a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:57.635014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 357/726] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:57.645800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dcd044-e69e-4070-9d7a-3c9aa8687a4d None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:57.660860 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f26a0-36c1-4a07-88a2-8ee4f0f58a8f None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:57.702762 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2547a2-2df8-458f-8fbd-c382e6fbdebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:57.703351 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2547a2-2df8-458f-8fbd-c382e6fbdebc None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:57.703635 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2547a2-2df8-458f-8fbd-c382e6fbdebc None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:57.703908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2547a2-2df8-458f-8fbd-c382e6fbdebc None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:57.704637 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2547a2-2df8-458f-8fbd-c382e6fbdebc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:57.704948 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2547a2-2df8-458f-8fbd-c382e6fbdebc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:57.710455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2547a2-2df8-458f-8fbd-c382e6fbdebc None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:57.736603 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 370/727] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:57.743551 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2547a2-2df8-458f-8fbd-c382e6fbdebc None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:57.750618 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:57.778229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f26a0-36c1-4a07-88a2-8ee4f0f58a8f None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:57.816417 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fd206-6108-42f3-b8a8-7e48759261f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:57.817311 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fd206-6108-42f3-b8a8-7e48759261f1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:57.817671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fd206-6108-42f3-b8a8-7e48759261f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:57.818066 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fd206-6108-42f3-b8a8-7e48759261f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:57.829909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26fd206-6108-42f3-b8a8-7e48759261f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:57.838491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-d26fd206-6108-42f3-b8a8-7e48759261f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:57.838491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:57.838491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:57.838491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:57.838491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:57.838491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:57.838491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:57.838491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:57.838491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:57.838491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:57.838491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:57.838491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:57.838491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:57.838491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:57.838491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:57.838491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:57.839453 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:57.839453 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:57.839453 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:57.839453 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:57.839453 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:57.839453 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:57.839453 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:57.839453 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:57.839453 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:57.839453 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:57.839453 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:57.839453 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:57.839453 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:57.839453 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:57.839453 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:57.839453 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:57.840296 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:57.840296 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:57.840296 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:57.840296 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:57.840296 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:57.840296 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:57.840296 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:57.840296 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:57.840296 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:57.840296 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:57.840296 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:58:57.840296 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:57.840296 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 358/728] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 15:58:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:58:57.846663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fd206-6108-42f3-b8a8-7e48759261f1 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:57.933690 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:57.959136 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fd206-6108-42f3-b8a8-7e48759261f1 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:58.003590 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:58.012047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2547a2-2df8-458f-8fbd-c382e6fbdebc None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:58.017282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9699bffd-4270-4637-b890-1e47d9d9064a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:58.018298 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699bffd-4270-4637-b890-1e47d9d9064a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:58.018791 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699bffd-4270-4637-b890-1e47d9d9064a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:58.019265 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699bffd-4270-4637-b890-1e47d9d9064a None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:58.020220 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9699bffd-4270-4637-b890-1e47d9d9064a None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:58.020711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9699bffd-4270-4637-b890-1e47d9d9064a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:58.030550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9699bffd-4270-4637-b890-1e47d9d9064a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:58.046857 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 359/729] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 15:58:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:58.052741 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9699bffd-4270-4637-b890-1e47d9d9064a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:58.058657 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6aa69e-aeec-4cba-8ac1-1b8db1766225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:58.059484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6aa69e-aeec-4cba-8ac1-1b8db1766225 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:58.059772 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6aa69e-aeec-4cba-8ac1-1b8db1766225 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:58.060051 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6aa69e-aeec-4cba-8ac1-1b8db1766225 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:58.069673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6aa69e-aeec-4cba-8ac1-1b8db1766225 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:58.071313 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6aa69e-aeec-4cba-8ac1-1b8db1766225 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:58.086943 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-af6aa69e-aeec-4cba-8ac1-1b8db1766225 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 15:58:58.086943 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:58.086943 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:58.086943 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:58.086943 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:58.086943 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:58.086943 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:58.086943 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:58.086943 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:58.086943 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:58.086943 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:58.086943 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:58.086943 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:58:58.086943 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:58:58.086943 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:58:58.086943 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:58:58.088430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:58.088430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:58.088430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:58:58.088430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:58.088430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:58:58.088430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:58.088430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:58.088430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:58.088430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:58.088430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:58.088430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:58.088430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:58.088430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:58.088430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:58.088430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:58.088430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:58.088430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:58.088430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:58.089530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:58.089530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:58.089530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:58:58.089530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:58:58.089530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:58:58.089530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:58:58.089530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:58:58.089530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:58:58.089530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 15:58:58.089530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:58.089530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 371/730] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 15:58:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:58:58.090254 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714f3e3-16bd-4648-a0f9-bece8ea33997 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:58.090254 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714f3e3-16bd-4648-a0f9-bece8ea33997 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:58.090404 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714f3e3-16bd-4648-a0f9-bece8ea33997 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:58.096642 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 372/731] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:58.097576 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1714f3e3-16bd-4648-a0f9-bece8ea33997 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:58.203914 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9699bffd-4270-4637-b890-1e47d9d9064a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:58.222816 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1714f3e3-16bd-4648-a0f9-bece8ea33997 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:58.245296 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:58.272071 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f0338-9356-4d96-b0f8-71fe1d19771d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:58.273223 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f0338-9356-4d96-b0f8-71fe1d19771d None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:58.273716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f0338-9356-4d96-b0f8-71fe1d19771d None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:58.274288 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f0338-9356-4d96-b0f8-71fe1d19771d None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:58.275316 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f0338-9356-4d96-b0f8-71fe1d19771d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:58.275424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da249b-27d6-4b7c-8896-1f42308ab87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:58.275914 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f0338-9356-4d96-b0f8-71fe1d19771d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:58.276854 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da249b-27d6-4b7c-8896-1f42308ab87c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:58.277463 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da249b-27d6-4b7c-8896-1f42308ab87c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:58.277976 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f0338-9356-4d96-b0f8-71fe1d19771d None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:58.278225 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da249b-27d6-4b7c-8896-1f42308ab87c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:58.303500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86da249b-27d6-4b7c-8896-1f42308ab87c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:58.304081 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 373/732] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jun 16 15:58:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:58.305690 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b145de7-683e-4292-9620-2bdfa00942fb None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:58.306185 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b145de7-683e-4292-9620-2bdfa00942fb None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:58.306658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b145de7-683e-4292-9620-2bdfa00942fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:58.307086 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86da249b-27d6-4b7c-8896-1f42308ab87c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:58.327314 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-8b145de7-683e-4292-9620-2bdfa00942fb None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:58.328837 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-8b145de7-683e-4292-9620-2bdfa00942fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:58.344851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-86da249b-27d6-4b7c-8896-1f42308ab87c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:58.344851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:58.344851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:58.344851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:58.344851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:58.344851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:58.344851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:58.344851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:58.344851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:58.344851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:58.344851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:58.344851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:58.344851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:58.344851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:58.344851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:58.345820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:58.345820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:58.345820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:58.345820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:58.345820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:58.345820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:58.345820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:58.345820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:58.345820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:58.345820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:58.345820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:58.345820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:58.345820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:58.345820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:58.345820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:58.345820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:58.345820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:58.346763 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:58.346763 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:58.346763 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:58.346763 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:58.347867 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 360/733] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:58.349332 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da249b-27d6-4b7c-8896-1f42308ab87c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:58.413804 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:58.416694 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 374/734] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:58.417953 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b145de7-683e-4292-9620-2bdfa00942fb None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:58.526199 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b145de7-683e-4292-9620-2bdfa00942fb None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:58.562386 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:58.569968 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da249b-27d6-4b7c-8896-1f42308ab87c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:58.570357 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600fd6d-d412-4c84-9b5d-8512a718d401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:58.571093 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600fd6d-d412-4c84-9b5d-8512a718d401 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:58.571378 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600fd6d-d412-4c84-9b5d-8512a718d401 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:58.571654 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600fd6d-d412-4c84-9b5d-8512a718d401 None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:58.572397 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f600fd6d-d412-4c84-9b5d-8512a718d401 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:58.572706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f600fd6d-d412-4c84-9b5d-8512a718d401 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:58.574157 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f600fd6d-d412-4c84-9b5d-8512a718d401 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:58.587957 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 375/735] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:58.589120 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600fd6d-d412-4c84-9b5d-8512a718d401 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:58.616512 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de729b3c-13e6-4fce-b6ef-11115ce9d613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:58.617576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de729b3c-13e6-4fce-b6ef-11115ce9d613 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:58.618113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de729b3c-13e6-4fce-b6ef-11115ce9d613 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:58.618624 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de729b3c-13e6-4fce-b6ef-11115ce9d613 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:58.708520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de729b3c-13e6-4fce-b6ef-11115ce9d613 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:58.709860 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600fd6d-d412-4c84-9b5d-8512a718d401 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:58.710438 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de729b3c-13e6-4fce-b6ef-11115ce9d613 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:58.749912 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c8c60-7746-4d16-a176-6916baa2d62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:58.750608 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c8c60-7746-4d16-a176-6916baa2d62e None admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:58.750895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c8c60-7746-4d16-a176-6916baa2d62e None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:58.751179 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c8c60-7746-4d16-a176-6916baa2d62e None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:58.751919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c8c60-7746-4d16-a176-6916baa2d62e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:58.753316 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c8c60-7746-4d16-a176-6916baa2d62e None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:58.765034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:58.782046 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 376/736] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jun 16 15:58:58 2022] POST /identity/v3/services => generated 251 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:58:58.782046 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c8c60-7746-4d16-a176-6916baa2d62e None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:58.836169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:58.851080 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 361/737] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:58.853301 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfca775-5a8a-46f5-bd64-97e4e6096bae None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:58.853801 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfca775-5a8a-46f5-bd64-97e4e6096bae None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:58.854381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfca775-5a8a-46f5-bd64-97e4e6096bae None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:58.855986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 362/738] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:58.856915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfca775-5a8a-46f5-bd64-97e4e6096bae None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:58.959701 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c8c60-7746-4d16-a176-6916baa2d62e None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:59.005554 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfca775-5a8a-46f5-bd64-97e4e6096bae None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:59.025336 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395c0c2-03c6-46c0-b624-22d8b841fd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:59.026714 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395c0c2-03c6-46c0-b624-22d8b841fd8d None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:59.027262 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395c0c2-03c6-46c0-b624-22d8b841fd8d None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:59.027780 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395c0c2-03c6-46c0-b624-22d8b841fd8d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:59.040342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4395c0c2-03c6-46c0-b624-22d8b841fd8d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:59.041841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4395c0c2-03c6-46c0-b624-22d8b841fd8d None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:59.065124 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-4395c0c2-03c6-46c0-b624-22d8b841fd8d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 15:58:59.065124 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:59.065124 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:59.065124 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:59.065124 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:59.065124 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:59.065124 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:59.065124 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:59.065124 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:59.065124 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:59.065124 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:59.065124 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:59.065124 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:58:59.065124 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:58:59.065124 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:58:59.065124 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:58:59.067162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:59.067162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:59.067162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:58:59.067162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:59.067162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:58:59.067162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:58:59.067162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:59.067162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:59.067162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:59.067162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:59.067162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:59.067162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:59.067162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:59.067162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:59.067162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:59.067162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:59.067162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:59.067162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:59.069211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:59.069211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:59.069211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:58:59.069211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:58:59.069211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:58:59.069211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:58:59.069211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:58:59.069211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:58:59.069211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 15:58:59.069211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:59.077512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 377/739] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jun 16 15:58:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:59.077512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b73f62-53e5-4597-b594-b4eaf5f409ec None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:59.077512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b73f62-53e5-4597-b594-b4eaf5f409ec None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:59.077512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b73f62-53e5-4597-b594-b4eaf5f409ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:59.090252 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3421896d-5f09-4981-b781-e8080dcb07d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:59.091571 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421896d-5f09-4981-b781-e8080dcb07d0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:59.092354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421896d-5f09-4981-b781-e8080dcb07d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:59.092845 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421896d-5f09-4981-b781-e8080dcb07d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:59.103228 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-a8b73f62-53e5-4597-b594-b4eaf5f409ec None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:59.105304 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-a8b73f62-53e5-4597-b594-b4eaf5f409ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:59.113691 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3421896d-5f09-4981-b781-e8080dcb07d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:59.117013 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3421896d-5f09-4981-b781-e8080dcb07d0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:59.129442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:59.145190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-3421896d-5f09-4981-b781-e8080dcb07d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:59.145190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:59.145190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:59.145190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:59.145190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:59.145190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:59.145190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:59.145190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:59.145190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:59.145190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:59.145190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:59.145190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:59.145190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:59.145190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:59.145190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:59.146229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:59.146229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:59.146229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:59.146229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:59.146229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:59.146229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:59.146229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:59.146229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:59.146229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:59.146229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:59.146229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:59.146229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:59.146229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:59.146229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:59.146229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:59.146229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:59.146229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:59.147518 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:59.147518 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:59.147518 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:59.147518 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:58:59.147518 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 363/740] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:59.148021 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3421896d-5f09-4981-b781-e8080dcb07d0 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:59.224306 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 378/741] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:59 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:59.226713 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0046f0-e52d-423f-a6dc-1c6782ce00fd None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:59.227258 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0046f0-e52d-423f-a6dc-1c6782ce00fd None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:59.227753 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0046f0-e52d-423f-a6dc-1c6782ce00fd None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:59.235212 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 379/742] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:58:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:59.235212 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0046f0-e52d-423f-a6dc-1c6782ce00fd None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:59.268988 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:59.289708 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3421896d-5f09-4981-b781-e8080dcb07d0 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:59.334734 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb94e964-f8b9-42b2-ade8-a302d3397bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:59.335809 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94e964-f8b9-42b2-ade8-a302d3397bea None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:59.336623 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94e964-f8b9-42b2-ade8-a302d3397bea None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:59.337064 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94e964-f8b9-42b2-ade8-a302d3397bea None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:59.338299 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb94e964-f8b9-42b2-ade8-a302d3397bea None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:59.338745 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb94e964-f8b9-42b2-ade8-a302d3397bea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:59.340837 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb94e964-f8b9-42b2-ade8-a302d3397bea None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:59.366546 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 364/743] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jun 16 15:58:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:59.368481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6fed62-9dac-4fcb-ba3b-44b498fa3415 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:59.369020 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6fed62-9dac-4fcb-ba3b-44b498fa3415 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:59.370039 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6fed62-9dac-4fcb-ba3b-44b498fa3415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:59.395554 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0d6fed62-9dac-4fcb-ba3b-44b498fa3415 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:59.397339 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0d6fed62-9dac-4fcb-ba3b-44b498fa3415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:59.408032 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0046f0-e52d-423f-a6dc-1c6782ce00fd None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:59.471673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c4c7e-8b62-4687-8013-b7ea79856450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:59.472770 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c4c7e-8b62-4687-8013-b7ea79856450 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:59.473227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c4c7e-8b62-4687-8013-b7ea79856450 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:59.473681 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c4c7e-8b62-4687-8013-b7ea79856450 None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:59.474920 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c4c7e-8b62-4687-8013-b7ea79856450 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:59.475449 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c4c7e-8b62-4687-8013-b7ea79856450 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:59.478143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c4c7e-8b62-4687-8013-b7ea79856450 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:59.484678 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:59.501183 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 380/744] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:59.502559 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c4c7e-8b62-4687-8013-b7ea79856450 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:59.523572 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 365/745] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:59 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:59.525332 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6fed62-9dac-4fcb-ba3b-44b498fa3415 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:59.611579 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:59.660615 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c4c7e-8b62-4687-8013-b7ea79856450 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:59.676441 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6fed62-9dac-4fcb-ba3b-44b498fa3415 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:59.701335 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb449a-70cb-48b5-aa31-a4062522c953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:59.702685 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb449a-70cb-48b5-aa31-a4062522c953 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:59.702968 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb449a-70cb-48b5-aa31-a4062522c953 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:59.703240 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb449a-70cb-48b5-aa31-a4062522c953 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:59.715619 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bb449a-70cb-48b5-aa31-a4062522c953 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:59.716276 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99c7c2-c51d-4a3e-8873-513ea9d14418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:59.716740 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99c7c2-c51d-4a3e-8873-513ea9d14418 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:59.717647 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99c7c2-c51d-4a3e-8873-513ea9d14418 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:59.717647 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99c7c2-c51d-4a3e-8873-513ea9d14418 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:59.723501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-90bb449a-70cb-48b5-aa31-a4062522c953 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:59.723501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:59.723501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:59.723501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:59.723501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:59.723501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:59.723501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:59.723501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:59.723501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:59.723501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:59.723501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:59.723501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:59.723501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:59.723501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:59.723501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:59.723501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:59.724461 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:59.724461 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:59.724461 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:59.724461 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:59.724461 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:59.724461 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:59.724461 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:59.724461 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:59.724461 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:59.724461 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:59.724461 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:59.724461 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:59.724461 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:59.724461 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:59.724461 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:59.724461 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:59.725900 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:59.725900 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:59.725900 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:59.725900 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:59.725900 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:59.725900 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:59.725900 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:59.725900 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:59.725900 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:59.725900 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:59.725900 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:59.725900 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:59.725900 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 381/746] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 15:58:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:59.726877 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb449a-70cb-48b5-aa31-a4062522c953 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:59.793441 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:59.816011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a99c7c2-c51d-4a3e-8873-513ea9d14418 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:59.817793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a99c7c2-c51d-4a3e-8873-513ea9d14418 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:59.875263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb449a-70cb-48b5-aa31-a4062522c953 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:59.909195 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:59.915561 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e4fd9-e2b6-45af-b9d3-96a59653af66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:59.916377 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e4fd9-e2b6-45af-b9d3-96a59653af66 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:59.916663 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e4fd9-e2b6-45af-b9d3-96a59653af66 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:59.916939 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e4fd9-e2b6-45af-b9d3-96a59653af66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:59.924888 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173e4fd9-e2b6-45af-b9d3-96a59653af66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:59.926430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173e4fd9-e2b6-45af-b9d3-96a59653af66 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:59.942371 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 366/747] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:59 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:59.942371 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99c7c2-c51d-4a3e-8873-513ea9d14418 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:59.943284 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-173e4fd9-e2b6-45af-b9d3-96a59653af66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:59.943284 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:59.943284 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:59.943284 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:59.943284 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:59.943284 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:59.943284 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:59.943284 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:59.943284 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:59.943284 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:59.943284 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:59.943284 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:59.943284 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:58:59.943284 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:58:59.943284 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:58:59.944907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:58:59.944907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:59.944907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:59.944907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:59.944907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:59.944907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:59.944907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:59.944907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:59.944907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:59.944907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:59.944907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:59.944907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:59.944907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:59.944907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:59.944907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:59.944907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:59.944907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:59.947085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:58:59.947085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:58:59.947085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:58:59.947085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:58:59.947085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:58:59.947085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:58:59.947085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:58:59.947085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:58:59.947085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 382/748] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:59.947085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e4fd9-e2b6-45af-b9d3-96a59653af66 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:00.059820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:00.072848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e4fd9-e2b6-45af-b9d3-96a59653af66 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:00.076329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99c7c2-c51d-4a3e-8873-513ea9d14418 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:00.115922 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944387d2-8ed7-4d85-9e4f-a00033a271f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:00.116588 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944387d2-8ed7-4d85-9e4f-a00033a271f3 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:00.116871 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944387d2-8ed7-4d85-9e4f-a00033a271f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:00.117145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944387d2-8ed7-4d85-9e4f-a00033a271f3 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:00.117865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944387d2-8ed7-4d85-9e4f-a00033a271f3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:00.119265 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944387d2-8ed7-4d85-9e4f-a00033a271f3 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:00.121512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d06cce-ba85-4769-954b-240b766059b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:00.122214 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d06cce-ba85-4769-954b-240b766059b4 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:00.122499 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d06cce-ba85-4769-954b-240b766059b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:00.122778 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d06cce-ba85-4769-954b-240b766059b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:00.124038 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d06cce-ba85-4769-954b-240b766059b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:00.124347 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d06cce-ba85-4769-954b-240b766059b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:00.125940 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d06cce-ba85-4769-954b-240b766059b4 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:00.143826 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 383/749] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:00.143826 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d06cce-ba85-4769-954b-240b766059b4 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:00.149799 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 367/750] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jun 16 15:58:59 2022] POST /identity/v3/services => generated 240 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:59:00.155139 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944387d2-8ed7-4d85-9e4f-a00033a271f3 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:00.236634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:00.309956 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d06cce-ba85-4769-954b-240b766059b4 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:00.327263 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944387d2-8ed7-4d85-9e4f-a00033a271f3 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:00.359697 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba021333-9bd1-407d-adfe-f6ffbe919363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:00.360385 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba021333-9bd1-407d-adfe-f6ffbe919363 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:00.360669 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba021333-9bd1-407d-adfe-f6ffbe919363 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:00.360946 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba021333-9bd1-407d-adfe-f6ffbe919363 None admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:00.361740 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba021333-9bd1-407d-adfe-f6ffbe919363 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:00.362085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba021333-9bd1-407d-adfe-f6ffbe919363 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:00.363747 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba021333-9bd1-407d-adfe-f6ffbe919363 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:00.382983 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 384/751] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:00.384430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba021333-9bd1-407d-adfe-f6ffbe919363 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:00.388295 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0c0b2-dd27-4df4-bb70-98c17cc724e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:00.389802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0c0b2-dd27-4df4-bb70-98c17cc724e5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:00.390118 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0c0b2-dd27-4df4-bb70-98c17cc724e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:00.391513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0c0b2-dd27-4df4-bb70-98c17cc724e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:00.408391 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc0c0b2-dd27-4df4-bb70-98c17cc724e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:00.409878 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc0c0b2-dd27-4df4-bb70-98c17cc724e5 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:00.413542 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:00.440407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-5cc0c0b2-dd27-4df4-bb70-98c17cc724e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:00.440407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:00.440407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:00.440407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:00.440407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:00.440407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:00.440407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:00.440407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:00.440407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:00.440407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:00.440407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:00.440407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:00.440407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:00.440407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:00.440407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:00.441439 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:00.441439 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:00.441439 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:00.441439 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:00.441439 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:00.441439 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:00.441439 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:00.441439 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:00.441439 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:00.441439 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:00.441439 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:00.441439 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:00.441439 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:00.441439 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:00.441439 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:00.441439 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:00.441439 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:00.442374 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:00.442374 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:00.442374 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:00.442374 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:00.443092 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 368/752] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:00.443998 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0c0b2-dd27-4df4-bb70-98c17cc724e5 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:00.529412 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba021333-9bd1-407d-adfe-f6ffbe919363 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:00.549528 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:00.577085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd657c5-1b76-4584-b581-7dee8614f7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:00.577907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd657c5-1b76-4584-b581-7dee8614f7fe None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:00.578242 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd657c5-1b76-4584-b581-7dee8614f7fe None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:00.578526 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd657c5-1b76-4584-b581-7dee8614f7fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:00.591429 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0c0b2-dd27-4df4-bb70-98c17cc724e5 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:00.595804 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd657c5-1b76-4584-b581-7dee8614f7fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:00.597626 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd657c5-1b76-4584-b581-7dee8614f7fe None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:00.622344 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-2dd657c5-1b76-4584-b581-7dee8614f7fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:00.622344 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:00.622344 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:00.622344 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:00.622344 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:00.622344 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:00.622344 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:00.622344 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:00.622344 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:00.622344 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:00.622344 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:00.622344 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:00.622344 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:00.622344 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:00.622344 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:00.624207 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:00.624207 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:00.624207 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:00.624207 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:00.624207 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:00.624207 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:00.624207 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:00.624207 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:00.624207 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:00.624207 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:00.624207 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:00.624207 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:00.624207 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:00.624207 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:00.624207 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:00.624207 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:00.624207 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:00.625116 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:00.625116 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:00.625116 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:00.625116 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:00.632586 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 385/753] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:00.632586 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd657c5-1b76-4584-b581-7dee8614f7fe None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:00.651762 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137f2e2-b4eb-4fb8-a682-e08274f86b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:00.652245 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137f2e2-b4eb-4fb8-a682-e08274f86b98 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:00.652532 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137f2e2-b4eb-4fb8-a682-e08274f86b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:00.652803 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137f2e2-b4eb-4fb8-a682-e08274f86b98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:00.653920 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0137f2e2-b4eb-4fb8-a682-e08274f86b98 None admin] RBAC: Adding query filter params (role.id=bbbc77ec645d4ff5a8bf29ce0377f578, scope.project.id=b1177a85a0de4276898daf4a24ff069f, user.id=0082e37ce54f49f7837601349508c89f) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:00.654277 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0137f2e2-b4eb-4fb8-a682-e08274f86b98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:00.657161 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0137f2e2-b4eb-4fb8-a682-e08274f86b98 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:00.669956 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 369/754] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:59:00 2022] GET /identity/v3/role_assignments?user.id=0082e37ce54f49f7837601349508c89f&scope.project.id=b1177a85a0de4276898daf4a24ff069f&role.id=bbbc77ec645d4ff5a8bf29ce0377f578 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:00.669956 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137f2e2-b4eb-4fb8-a682-e08274f86b98 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:00.817463 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:00.868388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd657c5-1b76-4584-b581-7dee8614f7fe None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:00.908108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137f2e2-b4eb-4fb8-a682-e08274f86b98 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:00.931991 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4198bf-4b66-4dc7-b068-7d57a8c98016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:00.932759 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4198bf-4b66-4dc7-b068-7d57a8c98016 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:00.933048 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4198bf-4b66-4dc7-b068-7d57a8c98016 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:00.933376 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4198bf-4b66-4dc7-b068-7d57a8c98016 None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:00.934208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4198bf-4b66-4dc7-b068-7d57a8c98016 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:00.934518 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4198bf-4b66-4dc7-b068-7d57a8c98016 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:00.936561 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4198bf-4b66-4dc7-b068-7d57a8c98016 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:00.950280 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc45ce-3669-4b2c-ae9c-7c1187ce9652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:00.951572 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc45ce-3669-4b2c-ae9c-7c1187ce9652 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:00.951572 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc45ce-3669-4b2c-ae9c-7c1187ce9652 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:00.956317 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc45ce-3669-4b2c-ae9c-7c1187ce9652 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:00.956317 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc45ce-3669-4b2c-ae9c-7c1187ce9652 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:00.956317 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc45ce-3669-4b2c-ae9c-7c1187ce9652 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:00.956317 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc45ce-3669-4b2c-ae9c-7c1187ce9652 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:00.965232 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 386/755] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:00.973186 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 370/756] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:00.974809 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4198bf-4b66-4dc7-b068-7d57a8c98016 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:00.979414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc45ce-3669-4b2c-ae9c-7c1187ce9652 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:01.055797 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:01.184319 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4198bf-4b66-4dc7-b068-7d57a8c98016 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:01.244904 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc45ce-3669-4b2c-ae9c-7c1187ce9652 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:01.252142 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3d5d5-7677-40a3-9df9-1b4d5d5bccdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:01.252970 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3d5d5-7677-40a3-9df9-1b4d5d5bccdb None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:01.253256 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3d5d5-7677-40a3-9df9-1b4d5d5bccdb None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:01.253536 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3d5d5-7677-40a3-9df9-1b4d5d5bccdb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:01.267786 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3d5d5-7677-40a3-9df9-1b4d5d5bccdb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:01.269404 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3d5d5-7677-40a3-9df9-1b4d5d5bccdb None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:01.284532 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:01.286618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-f1e3d5d5-7677-40a3-9df9-1b4d5d5bccdb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:01.286618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:01.286618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:01.286618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:01.286618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:01.286618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:01.286618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:01.286618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:01.286618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:01.286618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:01.286618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:01.286618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:01.286618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:01.286618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:01.286618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:01.287521 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:01.287521 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:01.287521 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:01.287521 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:01.287521 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:01.287521 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:01.287521 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:01.287521 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:01.287521 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:01.287521 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:01.287521 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:01.287521 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:01.287521 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:01.287521 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:01.287521 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:01.287521 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:01.287521 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:01.288620 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:01.288620 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:01.288620 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:01.288620 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:01.288620 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:01.288620 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:01.288620 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:01.288620 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:01.289066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 387/757] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:01.292855 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7badc968-ac1a-485b-8e83-c40f4dc44a81 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:01.292855 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7badc968-ac1a-485b-8e83-c40f4dc44a81 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:01.292855 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7badc968-ac1a-485b-8e83-c40f4dc44a81 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:01.293207 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 388/758] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:01.301579 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100206e-b9bf-4983-8dd7-87fa8fe0bed1 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:01.301579 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100206e-b9bf-4983-8dd7-87fa8fe0bed1 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:01.301579 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100206e-b9bf-4983-8dd7-87fa8fe0bed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:01.312590 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee123474-1ccf-4ab1-bb1d-ccccab5fa60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:01.313802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee123474-1ccf-4ab1-bb1d-ccccab5fa60b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:01.313802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee123474-1ccf-4ab1-bb1d-ccccab5fa60b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:01.314125 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee123474-1ccf-4ab1-bb1d-ccccab5fa60b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:01.320263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-4100206e-b9bf-4983-8dd7-87fa8fe0bed1 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:01.321709 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-4100206e-b9bf-4983-8dd7-87fa8fe0bed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:01.322264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee123474-1ccf-4ab1-bb1d-ccccab5fa60b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:01.323751 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee123474-1ccf-4ab1-bb1d-ccccab5fa60b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:01.339875 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-ee123474-1ccf-4ab1-bb1d-ccccab5fa60b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 15:59:01.339875 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:01.339875 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:01.339875 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:01.339875 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:01.339875 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:01.339875 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:01.339875 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:01.339875 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:01.339875 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:01.339875 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:01.339875 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:01.339875 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:01.339875 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:01.339875 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:01.339875 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:01.341123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:01.341123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:01.341123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:01.341123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:01.341123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:01.341123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:01.341123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:01.341123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:01.341123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:01.341123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:01.341123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:01.341123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:01.341123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:01.341123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:01.341123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:01.341123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:01.341123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:01.341123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:01.342333 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:01.342333 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:01.342333 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:01.342333 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:01.342333 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:01.342333 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:01.342333 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:01.342333 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:01.342333 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 15:59:01.342333 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:01.346361 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 371/759] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jun 16 15:59:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:01.347293 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee123474-1ccf-4ab1-bb1d-ccccab5fa60b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:01.406427 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 389/760] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:01.407625 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100206e-b9bf-4983-8dd7-87fa8fe0bed1 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:01.444018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:01.470245 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee123474-1ccf-4ab1-bb1d-ccccab5fa60b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:01.510231 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b1343-975b-40d7-a50f-a3deb14ada74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:01.510911 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b1343-975b-40d7-a50f-a3deb14ada74 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:01.511191 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b1343-975b-40d7-a50f-a3deb14ada74 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:01.511463 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b1343-975b-40d7-a50f-a3deb14ada74 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:01.512255 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b1343-975b-40d7-a50f-a3deb14ada74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:01.512563 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b1343-975b-40d7-a50f-a3deb14ada74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:01.514101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b1343-975b-40d7-a50f-a3deb14ada74 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:01.529020 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 372/761] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:01.529976 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b1343-975b-40d7-a50f-a3deb14ada74 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:01.544849 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100206e-b9bf-4983-8dd7-87fa8fe0bed1 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:01.584411 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d37a694-3ca3-4ed2-b23f-eefd72409832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:01.585585 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37a694-3ca3-4ed2-b23f-eefd72409832 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:01.586185 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37a694-3ca3-4ed2-b23f-eefd72409832 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:01.586870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37a694-3ca3-4ed2-b23f-eefd72409832 None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:01.587955 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d37a694-3ca3-4ed2-b23f-eefd72409832 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:01.588481 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d37a694-3ca3-4ed2-b23f-eefd72409832 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:01.590334 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d37a694-3ca3-4ed2-b23f-eefd72409832 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:01.590484 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:01.617963 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 390/762] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jun 16 15:59:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:01.617963 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d37a694-3ca3-4ed2-b23f-eefd72409832 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:01.664995 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b1343-975b-40d7-a50f-a3deb14ada74 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:01.700894 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:01.710612 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42023d2-26b5-4f82-8865-07fc12fb0954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:01.711726 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42023d2-26b5-4f82-8865-07fc12fb0954 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:01.712201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42023d2-26b5-4f82-8865-07fc12fb0954 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:01.712621 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42023d2-26b5-4f82-8865-07fc12fb0954 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:01.713541 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42023d2-26b5-4f82-8865-07fc12fb0954 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:01.715251 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42023d2-26b5-4f82-8865-07fc12fb0954 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:01.730875 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d37a694-3ca3-4ed2-b23f-eefd72409832 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:01.731793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-d42023d2-26b5-4f82-8865-07fc12fb0954 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 15:59:01.731793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:01.731793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 15:59:01.731793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 15:59:01.731793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 15:59:01.731793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 15:59:01.731793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 15:59:01.731793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:01.731793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 15:59:01.731793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:01.731793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:59:01.731793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:01.731793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:01.731793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:01.731793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:01.731793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:01.733598 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:01.733598 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:01.733598 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:01.733598 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:01.733598 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:01.733598 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:01.733598 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:01.733598 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 15:59:01.733598 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 15:59:01.733598 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 15:59:01.733598 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 15:59:01.733598 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:01.733598 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:01.733598 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:01.733598 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:01.733598 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:01.734546 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:01.734546 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:01.734546 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:01.734546 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:01.734546 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:01.734546 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:01.734546 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:01.734546 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:01.734546 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:01.734546 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:01.734546 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:01.734546 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 15:59:01.734546 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:01.734546 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 15:59:01.734546 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:01.737459 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 373/763] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jun 16 15:59:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:01.738758 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42023d2-26b5-4f82-8865-07fc12fb0954 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:01.782347 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe362a-36b8-4a4c-857b-a2e27970b598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:01.782881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe362a-36b8-4a4c-857b-a2e27970b598 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:01.783172 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe362a-36b8-4a4c-857b-a2e27970b598 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:01.783455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe362a-36b8-4a4c-857b-a2e27970b598 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:01.786131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe362a-36b8-4a4c-857b-a2e27970b598 None admin] RBAC: Adding query filter params (role.id=cfc67444e9944ffc87173be8c9245108, scope.project.id=b1177a85a0de4276898daf4a24ff069f, user.id=2c2e0e19880f4b048faf034dd38c7365) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:01.786131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe362a-36b8-4a4c-857b-a2e27970b598 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:01.786131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe362a-36b8-4a4c-857b-a2e27970b598 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:01.810462 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 391/764] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:59:01 2022] GET /identity/v3/role_assignments?user.id=2c2e0e19880f4b048faf034dd38c7365&scope.project.id=b1177a85a0de4276898daf4a24ff069f&role.id=cfc67444e9944ffc87173be8c9245108 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:01.810462 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe362a-36b8-4a4c-857b-a2e27970b598 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:01.953253 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42023d2-26b5-4f82-8865-07fc12fb0954 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:01.955060 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:01.992496 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe362a-36b8-4a4c-857b-a2e27970b598 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:02.039654 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c94858-480f-49cc-9187-20df768e1471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:02.040340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c94858-480f-49cc-9187-20df768e1471 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:02.040631 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c94858-480f-49cc-9187-20df768e1471 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:02.040913 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c94858-480f-49cc-9187-20df768e1471 None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:02.041639 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c94858-480f-49cc-9187-20df768e1471 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:02.041993 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c94858-480f-49cc-9187-20df768e1471 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:02.043619 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c94858-480f-49cc-9187-20df768e1471 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:02.050418 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 392/765] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jun 16 15:59:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:02.051430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab240bcf-f360-4881-b762-02574bd97aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:02.052642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab240bcf-f360-4881-b762-02574bd97aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:02.053316 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab240bcf-f360-4881-b762-02574bd97aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:02.054162 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab240bcf-f360-4881-b762-02574bd97aa8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:02.056560 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c94858-480f-49cc-9187-20df768e1471 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:02.060028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:02.069770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab240bcf-f360-4881-b762-02574bd97aa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:02.077953 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab240bcf-f360-4881-b762-02574bd97aa8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:02.109954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-ab240bcf-f360-4881-b762-02574bd97aa8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:02.109954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:02.109954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:02.109954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:02.109954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:02.109954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:02.109954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:02.109954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:02.109954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:02.109954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:02.109954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:02.109954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:02.109954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:02.109954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:02.109954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:02.110915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:02.110915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:02.110915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:02.110915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:02.110915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:02.110915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:02.110915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:02.110915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:02.110915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:02.110915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:02.110915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:02.110915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:02.110915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:02.110915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:02.110915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:02.110915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:02.110915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:02.111813 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:02.111813 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:02.111813 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:02.111813 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:02.114257 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 374/766] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:02.119650 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab240bcf-f360-4881-b762-02574bd97aa8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:02.174225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c94858-480f-49cc-9187-20df768e1471 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:02.213726 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09913a3b-d8ca-4ca4-9a44-9a7658653562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:02.214341 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09913a3b-d8ca-4ca4-9a44-9a7658653562 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:02.214632 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09913a3b-d8ca-4ca4-9a44-9a7658653562 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:02.214914 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09913a3b-d8ca-4ca4-9a44-9a7658653562 None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:02.215741 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09913a3b-d8ca-4ca4-9a44-9a7658653562 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:02.216166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09913a3b-d8ca-4ca4-9a44-9a7658653562 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:02.217800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09913a3b-d8ca-4ca4-9a44-9a7658653562 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:02.223956 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 393/767] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jun 16 15:59:02 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:02.229780 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09913a3b-d8ca-4ca4-9a44-9a7658653562 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:02.317301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:02.357599 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09913a3b-d8ca-4ca4-9a44-9a7658653562 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:02.381190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab240bcf-f360-4881-b762-02574bd97aa8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:02.397221 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37f321-7172-4ebe-a033-c5c483c8faaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:02.397896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37f321-7172-4ebe-a033-c5c483c8faaa None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:02.398209 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37f321-7172-4ebe-a033-c5c483c8faaa None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:02.398487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37f321-7172-4ebe-a033-c5c483c8faaa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:02.399288 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37f321-7172-4ebe-a033-c5c483c8faaa None admin] RBAC: Adding query filter params (interface=public, service_id=31c4fe092f774bf5984ab248f535a149, region_id=RegionOne) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:02.399748 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37f321-7172-4ebe-a033-c5c483c8faaa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:02.401127 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37f321-7172-4ebe-a033-c5c483c8faaa None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:02.411055 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 394/768] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jun 16 15:59:02 2022] GET /identity/v3/endpoints?service_id=31c4fe092f774bf5984ab248f535a149&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:02.416354 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37f321-7172-4ebe-a033-c5c483c8faaa None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:02.442328 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5a29e-d1d1-4730-a272-9a0e4dde7a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:02.443000 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5a29e-d1d1-4730-a272-9a0e4dde7a58 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:02.443284 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5a29e-d1d1-4730-a272-9a0e4dde7a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:02.443555 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5a29e-d1d1-4730-a272-9a0e4dde7a58 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:02.444289 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc5a29e-d1d1-4730-a272-9a0e4dde7a58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:02.444591 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc5a29e-d1d1-4730-a272-9a0e4dde7a58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:02.448128 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc5a29e-d1d1-4730-a272-9a0e4dde7a58 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:02.470554 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 375/769] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:02.475773 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5a29e-d1d1-4730-a272-9a0e4dde7a58 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:02.507314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:02.532066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37f321-7172-4ebe-a033-c5c483c8faaa None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:02.582640 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:02.587503 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb5110-2cd1-4bba-8ca4-794ca854c369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:02.588351 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb5110-2cd1-4bba-8ca4-794ca854c369 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:02.588821 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb5110-2cd1-4bba-8ca4-794ca854c369 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:02.589173 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb5110-2cd1-4bba-8ca4-794ca854c369 None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:02.590011 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb5110-2cd1-4bba-8ca4-794ca854c369 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:02.591514 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb5110-2cd1-4bba-8ca4-794ca854c369 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:02.598799 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 395/770] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jun 16 15:59:02 2022] GET /identity/v3/services => generated 772 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:02.719923 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5a29e-d1d1-4730-a272-9a0e4dde7a58 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:02.732798 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1d0d4-f1a0-4eee-a808-de295d6ea460 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:02.732993 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1d0d4-f1a0-4eee-a808-de295d6ea460 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:02.733159 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1d0d4-f1a0-4eee-a808-de295d6ea460 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:02.736811 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 396/771] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:59:02.742206 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18f749-73c0-4aee-bac4-e26e2f3dd42b None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:02.742499 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18f749-73c0-4aee-bac4-e26e2f3dd42b None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:02.742785 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18f749-73c0-4aee-bac4-e26e2f3dd42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:02.763800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-5d18f749-73c0-4aee-bac4-e26e2f3dd42b None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:02.765587 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-5d18f749-73c0-4aee-bac4-e26e2f3dd42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:02.814991 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1f5e0-8489-479b-977b-b4a237a3e43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:02.816054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1f5e0-8489-479b-977b-b4a237a3e43b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:02.816554 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1f5e0-8489-479b-977b-b4a237a3e43b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:02.816938 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1f5e0-8489-479b-977b-b4a237a3e43b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:02.827278 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1f5e0-8489-479b-977b-b4a237a3e43b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:02.829799 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1f5e0-8489-479b-977b-b4a237a3e43b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:02.849079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-cda1f5e0-8489-479b-977b-b4a237a3e43b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:02.849079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:02.849079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:02.849079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:02.849079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:02.849079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:02.849079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:02.849079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:02.849079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:02.849079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:02.849079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:02.849079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:02.849079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:02.849079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:02.849079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:02.850095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:02.850095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:02.850095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:02.850095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:02.850095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:02.850095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:02.850095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:02.850095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:02.850095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:02.850095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:02.850095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:02.850095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:02.850095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:02.850095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:02.850095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:02.850095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:02.850095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:02.850987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:02.850987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:02.850987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:02.850987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:02.850987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:02.850987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:02.850987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:02.850987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:02.851462 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 376/772] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:02.858255 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1f5e0-8489-479b-977b-b4a237a3e43b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:02.874407 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 397/773] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 15:59:02.892031 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18f749-73c0-4aee-bac4-e26e2f3dd42b None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:02.946060 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:02.973913 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1f5e0-8489-479b-977b-b4a237a3e43b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:03.001128 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18f749-73c0-4aee-bac4-e26e2f3dd42b None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:03.024118 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02efcc-bbe5-437a-b9b5-c2599c6c944d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:03.024719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02efcc-bbe5-437a-b9b5-c2599c6c944d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:03.025006 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02efcc-bbe5-437a-b9b5-c2599c6c944d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:03.025282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02efcc-bbe5-437a-b9b5-c2599c6c944d None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:03.026197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02efcc-bbe5-437a-b9b5-c2599c6c944d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:03.026423 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02efcc-bbe5-437a-b9b5-c2599c6c944d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:03.027904 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02efcc-bbe5-437a-b9b5-c2599c6c944d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:03.039927 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984ae5c-deef-4671-ab5e-67ca8f842709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:03.041080 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984ae5c-deef-4671-ab5e-67ca8f842709 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:03.041567 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984ae5c-deef-4671-ab5e-67ca8f842709 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:03.042092 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984ae5c-deef-4671-ab5e-67ca8f842709 None admin] PATH_INFO: `/v3/services/network` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:03.042966 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984ae5c-deef-4671-ab5e-67ca8f842709 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:03.044619 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984ae5c-deef-4671-ab5e-67ca8f842709 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:03.046433 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:03.055046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 377/774] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:03.064701 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02efcc-bbe5-437a-b9b5-c2599c6c944d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:03.072732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-1984ae5c-deef-4671-ab5e-67ca8f842709 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 15:59:03.072732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:03.072732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 15:59:03.072732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 15:59:03.072732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 15:59:03.072732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 15:59:03.072732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 15:59:03.072732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:03.072732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 15:59:03.072732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:03.072732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:59:03.072732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:03.072732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:03.072732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:03.072732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:03.072732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:03.072732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:03.073752 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:03.073752 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:03.073752 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:03.073752 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:03.073752 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:03.073752 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:03.073752 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 15:59:03.073752 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 15:59:03.073752 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 15:59:03.073752 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 15:59:03.073752 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:03.073752 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:03.073752 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:03.073752 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:03.073752 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:03.073752 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:03.074638 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:03.074638 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:03.074638 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:03.074638 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:03.074638 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:03.074638 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:03.074638 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:03.074638 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:03.074638 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:03.074638 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:03.074638 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 15:59:03.074638 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:03.074638 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 15:59:03.074638 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:03.084091 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 398/775] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:02 2022] GET /identity/v3/services/network => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:03.088415 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984ae5c-deef-4671-ab5e-67ca8f842709 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:03.199355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984ae5c-deef-4671-ab5e-67ca8f842709 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:03.226826 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02efcc-bbe5-437a-b9b5-c2599c6c944d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:03.238967 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36531bf9-44e6-4dcf-90b7-71754ce5a732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:03.240041 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36531bf9-44e6-4dcf-90b7-71754ce5a732 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:03.240536 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36531bf9-44e6-4dcf-90b7-71754ce5a732 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:03.241099 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36531bf9-44e6-4dcf-90b7-71754ce5a732 None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:03.242052 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36531bf9-44e6-4dcf-90b7-71754ce5a732 None admin] RBAC: Adding query filter params (name=network) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:03.242563 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36531bf9-44e6-4dcf-90b7-71754ce5a732 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:03.244274 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36531bf9-44e6-4dcf-90b7-71754ce5a732 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:03.256034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 399/776] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jun 16 15:59:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:03.257868 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36531bf9-44e6-4dcf-90b7-71754ce5a732 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:03.308788 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13905fba-d145-4dab-81fd-da25610aa701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:03.309162 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13905fba-d145-4dab-81fd-da25610aa701 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:03.309478 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13905fba-d145-4dab-81fd-da25610aa701 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:03.309759 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13905fba-d145-4dab-81fd-da25610aa701 None admin] PATH_INFO: `/v3/projects/b1177a85a0de4276898daf4a24ff069f/users/c3645b1a0be34c01ac87c734fcd85ff6/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:03.318954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13905fba-d145-4dab-81fd-da25610aa701 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1177a85a0de4276898daf4a24ff069f, user_id=c3645b1a0be34c01ac87c734fcd85ff6, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:03.320464 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13905fba-d145-4dab-81fd-da25610aa701 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:03.323047 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:03.328364 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:03.348734 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 378/777] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 15:59:03 2022] PUT /identity/v3/projects/b1177a85a0de4276898daf4a24ff069f/users/c3645b1a0be34c01ac87c734fcd85ff6/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:59:03.358164 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56825085-73a6-4ea0-b442-a9254e945d40 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:03.358553 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56825085-73a6-4ea0-b442-a9254e945d40 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:03.358738 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56825085-73a6-4ea0-b442-a9254e945d40 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:03.360903 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 379/778] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:03.366224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65296670-2e0a-4c15-9dd7-8982966f986a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:03.366574 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65296670-2e0a-4c15-9dd7-8982966f986a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:03.366796 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65296670-2e0a-4c15-9dd7-8982966f986a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:03.393702 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36531bf9-44e6-4dcf-90b7-71754ce5a732 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:03.415285 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-65296670-2e0a-4c15-9dd7-8982966f986a None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:03.416857 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-65296670-2e0a-4c15-9dd7-8982966f986a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:03.450738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c1ec5-ba81-4bac-bb2e-3f230234ab5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:03.451998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c1ec5-ba81-4bac-bb2e-3f230234ab5a None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:03.452288 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c1ec5-ba81-4bac-bb2e-3f230234ab5a None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:03.452570 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c1ec5-ba81-4bac-bb2e-3f230234ab5a None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:03.453301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c1ec5-ba81-4bac-bb2e-3f230234ab5a None admin] RBAC: Adding query filter params (type=network) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:03.458875 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c1ec5-ba81-4bac-bb2e-3f230234ab5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:03.460553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c1ec5-ba81-4bac-bb2e-3f230234ab5a None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:03.471181 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 400/779] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jun 16 15:59:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:03.478413 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c1ec5-ba81-4bac-bb2e-3f230234ab5a None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:03.571626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 380/780] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:03.596778 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65296670-2e0a-4c15-9dd7-8982966f986a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:03.651528 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c1ec5-ba81-4bac-bb2e-3f230234ab5a None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:03.687693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:03.716742 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af33a84-846e-42c2-b939-2f67e8b7a02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:03.717823 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af33a84-846e-42c2-b939-2f67e8b7a02a None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:03.718534 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af33a84-846e-42c2-b939-2f67e8b7a02a None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:03.719043 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af33a84-846e-42c2-b939-2f67e8b7a02a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:03.722868 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af33a84-846e-42c2-b939-2f67e8b7a02a None admin] RBAC: Adding query filter params (interface=public, service_id=475354c5712d40f9bc8fdccd21d97a0b, region_id=RegionOne) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:03.723422 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af33a84-846e-42c2-b939-2f67e8b7a02a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:03.725049 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af33a84-846e-42c2-b939-2f67e8b7a02a None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:03.738083 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 401/781] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jun 16 15:59:03 2022] GET /identity/v3/endpoints?service_id=475354c5712d40f9bc8fdccd21d97a0b&interface=public®ion_id=RegionOne => generated 194 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:03.743556 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af33a84-846e-42c2-b939-2f67e8b7a02a None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:03.750776 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65296670-2e0a-4c15-9dd7-8982966f986a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:03.810216 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:03.833371 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a056d9-6c83-4bdb-906e-3462236c296d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:03.834180 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a056d9-6c83-4bdb-906e-3462236c296d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:03.834463 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a056d9-6c83-4bdb-906e-3462236c296d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:03.834734 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a056d9-6c83-4bdb-906e-3462236c296d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:03.860597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a056d9-6c83-4bdb-906e-3462236c296d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:03.874115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-98a056d9-6c83-4bdb-906e-3462236c296d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:59:03.874115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:03.874115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:03.874115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:03.874115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:03.874115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:03.874115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:03.874115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:03.874115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:03.874115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:03.874115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:03.874115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:03.874115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:03.874115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:03.874115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:03.874115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:03.875653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:03.875653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:03.875653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:03.875653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:03.875653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:03.875653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:03.875653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:03.875653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:03.875653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:03.875653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:03.875653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:03.875653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:03.875653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:03.875653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:03.875653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:03.875653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:03.876496 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:03.876496 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:03.876496 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:03.876496 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:03.876496 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:03.876496 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:03.876496 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:03.876496 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:03.876496 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:03.876496 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:03.876496 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:59:03.876496 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:03.885174 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 381/782] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 15:59:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:03.886255 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a056d9-6c83-4bdb-906e-3462236c296d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:03.947402 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af33a84-846e-42c2-b939-2f67e8b7a02a None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:03.993779 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb5feab-3057-410b-8814-9ffc9d86ae4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:03.994511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb5feab-3057-410b-8814-9ffc9d86ae4e None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:03.994801 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb5feab-3057-410b-8814-9ffc9d86ae4e None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:03.995109 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb5feab-3057-410b-8814-9ffc9d86ae4e None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:03.995868 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb5feab-3057-410b-8814-9ffc9d86ae4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:03.997357 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb5feab-3057-410b-8814-9ffc9d86ae4e None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:04.015935 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 402/783] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jun 16 15:59:03 2022] GET /identity/v3/services => generated 772 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:04.049275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a056d9-6c83-4bdb-906e-3462236c296d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:04.094342 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c1b05e-516f-4630-9015-6db9a5f4c765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:04.095013 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1b05e-516f-4630-9015-6db9a5f4c765 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:04.095309 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1b05e-516f-4630-9015-6db9a5f4c765 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:04.095593 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1b05e-516f-4630-9015-6db9a5f4c765 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:04.096386 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c1b05e-516f-4630-9015-6db9a5f4c765 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:04.096700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c1b05e-516f-4630-9015-6db9a5f4c765 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:04.098171 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c1b05e-516f-4630-9015-6db9a5f4c765 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:04.108219 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 382/784] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:04.113905 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb5feab-3057-410b-8814-9ffc9d86ae4e None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:04.196333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:04.273719 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb5feab-3057-410b-8814-9ffc9d86ae4e None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:04.318525 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf30429-d2fb-4a56-b952-c300d149fa5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:04.319430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf30429-d2fb-4a56-b952-c300d149fa5f None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:04.319712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf30429-d2fb-4a56-b952-c300d149fa5f None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:04.319989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf30429-d2fb-4a56-b952-c300d149fa5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:04.333401 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf30429-d2fb-4a56-b952-c300d149fa5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:04.334931 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf30429-d2fb-4a56-b952-c300d149fa5f None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:04.353823 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-acf30429-d2fb-4a56-b952-c300d149fa5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:04.353823 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:04.353823 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:04.353823 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:04.353823 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:04.353823 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:04.353823 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:04.353823 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:04.353823 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:04.353823 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:04.353823 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:04.353823 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:04.353823 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:04.353823 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:04.353823 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:04.354841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:04.354841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:04.354841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:04.354841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:04.354841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:04.354841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:04.354841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:04.354841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:04.354841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:04.354841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:04.354841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:04.354841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:04.354841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:04.354841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:04.354841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:04.354841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:04.354841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:04.355735 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:04.355735 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:04.355735 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:04.355735 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:04.355735 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 403/785] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:04.362397 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c1b05e-516f-4630-9015-6db9a5f4c765 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:04.456519 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:04.567352 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c1b05e-516f-4630-9015-6db9a5f4c765 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:04.620675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f37a0-4b4c-4420-8b5f-e5afdbc0961c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:04.621269 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f37a0-4b4c-4420-8b5f-e5afdbc0961c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:04.621548 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f37a0-4b4c-4420-8b5f-e5afdbc0961c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:04.621885 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f37a0-4b4c-4420-8b5f-e5afdbc0961c None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:04.622636 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3f37a0-4b4c-4420-8b5f-e5afdbc0961c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:04.622988 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3f37a0-4b4c-4420-8b5f-e5afdbc0961c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:04.625198 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3f37a0-4b4c-4420-8b5f-e5afdbc0961c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:04.648341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 383/786] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:04.653536 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf30429-d2fb-4a56-b952-c300d149fa5f None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:04.659594 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa3e5f-8a06-45ee-be55-fe032a138827 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:04.659948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa3e5f-8a06-45ee-be55-fe032a138827 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:04.660226 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa3e5f-8a06-45ee-be55-fe032a138827 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:04.661835 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 384/787] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:04.672830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06d661-1514-45f9-b3d8-53b1bdf4bde5 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:04.673140 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06d661-1514-45f9-b3d8-53b1bdf4bde5 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:04.673382 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06d661-1514-45f9-b3d8-53b1bdf4bde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:04.876000 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0c06d661-1514-45f9-b3d8-53b1bdf4bde5 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:04.877242 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0c06d661-1514-45f9-b3d8-53b1bdf4bde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:04.902375 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:04.911615 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf30429-d2fb-4a56-b952-c300d149fa5f None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:04.984016 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e70d2-1e58-41bb-8f19-3057e5df5311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:04.984926 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e70d2-1e58-41bb-8f19-3057e5df5311 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:04.985210 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e70d2-1e58-41bb-8f19-3057e5df5311 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:04.985482 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e70d2-1e58-41bb-8f19-3057e5df5311 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:04.994488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e70d2-1e58-41bb-8f19-3057e5df5311 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:04.995997 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e70d2-1e58-41bb-8f19-3057e5df5311 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:05.012977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-8c2e70d2-1e58-41bb-8f19-3057e5df5311 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 15:59:05.012977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:05.012977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:05.012977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:05.012977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:05.012977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:05.012977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:05.012977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:05.012977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:05.012977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:05.012977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:05.012977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:05.012977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:05.012977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:05.012977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:05.012977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:05.014227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:05.014227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:05.014227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:05.014227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:05.014227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:05.014227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:05.014227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:05.014227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:05.014227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:05.014227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:05.014227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:05.014227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:05.014227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:05.014227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:05.014227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:05.014227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:05.014227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:05.014227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:05.015337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:05.015337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:05.015337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:05.015337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:05.015337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:05.015337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:05.015337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:05.015337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:05.015337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 15:59:05.015337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:05.015337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 404/788] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 15:59:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:05.020479 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e70d2-1e58-41bb-8f19-3057e5df5311 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:05.035692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 385/789] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 15:59:05.077617 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06d661-1514-45f9-b3d8-53b1bdf4bde5 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:05.153819 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e70d2-1e58-41bb-8f19-3057e5df5311 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:05.171704 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:05.197662 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46297e-c92e-4bc6-881a-5d28274b41f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:05.198391 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46297e-c92e-4bc6-881a-5d28274b41f9 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:05.198673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46297e-c92e-4bc6-881a-5d28274b41f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:05.198947 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46297e-c92e-4bc6-881a-5d28274b41f9 None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:05.199794 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f46297e-c92e-4bc6-881a-5d28274b41f9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:05.200101 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f46297e-c92e-4bc6-881a-5d28274b41f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:05.202190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f46297e-c92e-4bc6-881a-5d28274b41f9 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:05.233257 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 405/790] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jun 16 15:59:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:05.240559 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46297e-c92e-4bc6-881a-5d28274b41f9 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:05.264915 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:05.332306 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06d661-1514-45f9-b3d8-53b1bdf4bde5 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:05.455484 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0bc76c-e6aa-4f90-ab3a-76959df49f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:05.456809 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bc76c-e6aa-4f90-ab3a-76959df49f14 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:05.457320 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bc76c-e6aa-4f90-ab3a-76959df49f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:05.457828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bc76c-e6aa-4f90-ab3a-76959df49f14 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:05.494430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0bc76c-e6aa-4f90-ab3a-76959df49f14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:05.502993 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-1c0bc76c-e6aa-4f90-ab3a-76959df49f14 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:05.502993 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:05.502993 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:05.502993 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:05.502993 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:05.502993 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:05.502993 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:05.502993 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:05.502993 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:05.502993 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:05.502993 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:05.502993 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:05.502993 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:05.502993 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:05.502993 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:05.502993 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:05.504093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:05.504093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:05.504093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:05.504093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:05.504093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:05.504093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:05.504093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:05.504093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:05.504093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:05.504093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:05.504093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:05.504093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:05.504093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:05.504093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:05.504093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:05.504093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:05.504945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:05.504945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:05.504945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:05.504945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:05.504945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:05.504945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:05.504945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:05.504945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:05.504945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:05.504945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:05.504945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:05.504945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:05.504945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 386/791] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 15:59:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 427 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:05.515754 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0bc76c-e6aa-4f90-ab3a-76959df49f14 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:05.525175 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46297e-c92e-4bc6-881a-5d28274b41f9 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:05.644130 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0bc76c-e6aa-4f90-ab3a-76959df49f14 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:05.645096 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d1e0a-a83e-413f-80f1-b513bdc1e0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:05.646152 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d1e0a-a83e-413f-80f1-b513bdc1e0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:05.646317 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:05.646469 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d1e0a-a83e-413f-80f1-b513bdc1e0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:05.646711 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d1e0a-a83e-413f-80f1-b513bdc1e0b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:05.675085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d1e0a-a83e-413f-80f1-b513bdc1e0b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:05.676854 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d1e0a-a83e-413f-80f1-b513bdc1e0b6 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:05.718599 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-bc1d1e0a-a83e-413f-80f1-b513bdc1e0b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:05.718599 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:05.718599 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:05.718599 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:05.718599 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:05.718599 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:05.718599 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:05.718599 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:05.718599 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:05.718599 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:05.718599 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:05.718599 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:05.718599 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:05.718599 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:05.718599 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:05.720167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:05.720167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:05.720167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:05.720167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:05.720167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:05.720167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:05.720167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:05.720167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:05.720167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:05.720167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:05.720167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:05.720167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:05.720167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:05.720167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:05.720167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:05.720167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:05.720167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:05.721710 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:05.721710 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:05.721710 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:05.721710 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:05.721710 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 406/792] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 477 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:05.723560 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d4455-1281-45e4-81f5-79e63c9927b8 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:05.723850 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d4455-1281-45e4-81f5-79e63c9927b8 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:05.724136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d4455-1281-45e4-81f5-79e63c9927b8 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:05.725098 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f2e6d-da7b-4813-9100-4d876970b933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:05.725960 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 407/793] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:05.726535 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f2e6d-da7b-4813-9100-4d876970b933 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:05.727013 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f2e6d-da7b-4813-9100-4d876970b933 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:05.727469 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f2e6d-da7b-4813-9100-4d876970b933 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:05.728606 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f2e6d-da7b-4813-9100-4d876970b933 None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:05.729216 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f2e6d-da7b-4813-9100-4d876970b933 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:05.731211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d4455-1281-45e4-81f5-79e63c9927b8 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:05.732319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f2e6d-da7b-4813-9100-4d876970b933 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:05.746642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 387/794] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 15:59:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:05.749832 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f29e74-2406-4347-9470-0d31fdeeba5c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:05.750753 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f29e74-2406-4347-9470-0d31fdeeba5c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:05.751560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f29e74-2406-4347-9470-0d31fdeeba5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:05.793983 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-43f29e74-2406-4347-9470-0d31fdeeba5c None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:05.793983 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-43f29e74-2406-4347-9470-0d31fdeeba5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:05.828157 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:05.893140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d4455-1281-45e4-81f5-79e63c9927b8 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:05.894980 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 388/795] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:05.896405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f29e74-2406-4347-9470-0d31fdeeba5c None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:05.978328 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6963e3-2df5-4fbf-803b-df882559e0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:05.979653 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6963e3-2df5-4fbf-803b-df882559e0de None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:05.980161 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6963e3-2df5-4fbf-803b-df882559e0de None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:05.980639 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6963e3-2df5-4fbf-803b-df882559e0de None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:05.981772 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6963e3-2df5-4fbf-803b-df882559e0de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:05.982318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6963e3-2df5-4fbf-803b-df882559e0de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:05.985624 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6963e3-2df5-4fbf-803b-df882559e0de None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:06.002250 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 408/796] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:06.003547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6963e3-2df5-4fbf-803b-df882559e0de None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:06.033967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:06.060340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f29e74-2406-4347-9470-0d31fdeeba5c None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:06.121717 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15aa53e-079a-4da3-98e4-0e3ab3777877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:06.124059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15aa53e-079a-4da3-98e4-0e3ab3777877 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:06.124567 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15aa53e-079a-4da3-98e4-0e3ab3777877 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:06.125155 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15aa53e-079a-4da3-98e4-0e3ab3777877 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:06.141030 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15aa53e-079a-4da3-98e4-0e3ab3777877 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:06.142947 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15aa53e-079a-4da3-98e4-0e3ab3777877 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:06.166358 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6963e3-2df5-4fbf-803b-df882559e0de None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:06.167808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-c15aa53e-079a-4da3-98e4-0e3ab3777877 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:06.167808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:06.167808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:06.167808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:06.167808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:06.167808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:06.167808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:06.167808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:06.167808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:06.167808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:06.167808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:06.167808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:06.167808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:06.167808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:06.167808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:06.168671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:06.168671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:06.168671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:06.168671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:06.168671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:06.168671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:06.168671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:06.168671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:06.168671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:06.168671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:06.168671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:06.168671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:06.168671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:06.168671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:06.168671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:06.168671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:06.168671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:06.169581 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:06.169581 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:06.169581 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:06.169581 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:06.172288 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 389/797] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:06.173313 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15aa53e-079a-4da3-98e4-0e3ab3777877 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:06.210464 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:06.222904 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94abc794-daa1-4394-84a3-da568428a823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:06.224015 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94abc794-daa1-4394-84a3-da568428a823 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:06.224459 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94abc794-daa1-4394-84a3-da568428a823 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:06.224893 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94abc794-daa1-4394-84a3-da568428a823 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:06.225767 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94abc794-daa1-4394-84a3-da568428a823 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:06.227537 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94abc794-daa1-4394-84a3-da568428a823 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:06.241494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-94abc794-daa1-4394-84a3-da568428a823 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 15:59:06.241494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:06.241494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 15:59:06.241494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 15:59:06.241494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 15:59:06.241494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 15:59:06.241494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 15:59:06.241494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:06.241494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 15:59:06.241494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:06.241494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:59:06.241494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:06.241494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:06.241494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:06.241494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:06.241494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:06.242734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:06.242734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:06.242734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:06.242734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:06.242734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:06.242734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:06.242734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:06.242734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 15:59:06.242734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 15:59:06.242734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 15:59:06.242734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 15:59:06.242734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:06.242734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:06.242734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:06.242734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:06.242734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:06.243608 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:06.243608 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:06.243608 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:06.243608 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:06.243608 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:06.243608 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:06.243608 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:06.243608 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:06.243608 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:06.243608 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:06.243608 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:06.243608 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 15:59:06.243608 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:06.243608 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 15:59:06.243608 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:06.249970 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 409/798] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jun 16 15:59:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:06.251242 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94abc794-daa1-4394-84a3-da568428a823 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:06.309682 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15aa53e-079a-4da3-98e4-0e3ab3777877 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:06.358675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54eb0a-be93-4f67-a10a-b7d982f4d3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:06.359665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54eb0a-be93-4f67-a10a-b7d982f4d3df None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:06.360322 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54eb0a-be93-4f67-a10a-b7d982f4d3df None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:06.360614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54eb0a-be93-4f67-a10a-b7d982f4d3df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:06.368295 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54eb0a-be93-4f67-a10a-b7d982f4d3df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:06.368939 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94abc794-daa1-4394-84a3-da568428a823 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:06.369744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54eb0a-be93-4f67-a10a-b7d982f4d3df None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:06.374190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:06.388165 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-6f54eb0a-be93-4f67-a10a-b7d982f4d3df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:06.388165 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:06.388165 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:06.388165 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:06.388165 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:06.388165 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:06.388165 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:06.388165 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:06.388165 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:06.388165 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:06.388165 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:06.388165 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:06.388165 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:06.388165 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:06.388165 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:06.389218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:06.389218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:06.389218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:06.389218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:06.389218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:06.389218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:06.389218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:06.389218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:06.389218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:06.389218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:06.389218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:06.389218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:06.389218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:06.389218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:06.389218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:06.389218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:06.389218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:06.390889 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:06.390889 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:06.390889 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:06.390889 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:06.390889 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:06.390889 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:06.390889 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:06.390889 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:06.395010 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 390/799] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:06.395823 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54eb0a-be93-4f67-a10a-b7d982f4d3df None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:06.419413 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787037e-474c-4837-969b-c6aefc5966ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:06.419866 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787037e-474c-4837-969b-c6aefc5966ad None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:06.419959 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787037e-474c-4837-969b-c6aefc5966ad None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:06.420391 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787037e-474c-4837-969b-c6aefc5966ad None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:06.422627 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787037e-474c-4837-969b-c6aefc5966ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:06.422627 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787037e-474c-4837-969b-c6aefc5966ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:06.424381 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787037e-474c-4837-969b-c6aefc5966ad None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:06.457970 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 410/800] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:06.457970 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8239ff-a913-411d-b7d8-d8627574b709 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:06.457970 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8239ff-a913-411d-b7d8-d8627574b709 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:06.457970 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8239ff-a913-411d-b7d8-d8627574b709 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:06.457970 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 411/801] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:06.459074 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8239ff-a913-411d-b7d8-d8627574b709 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:06.524657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54eb0a-be93-4f67-a10a-b7d982f4d3df None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:06.548324 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:06.572608 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfa14c-a991-415b-bc97-2dc502caaf55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:06.573283 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfa14c-a991-415b-bc97-2dc502caaf55 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:06.574266 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfa14c-a991-415b-bc97-2dc502caaf55 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:06.574266 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfa14c-a991-415b-bc97-2dc502caaf55 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:06.574664 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfa14c-a991-415b-bc97-2dc502caaf55 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:06.574904 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfa14c-a991-415b-bc97-2dc502caaf55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:06.576262 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfa14c-a991-415b-bc97-2dc502caaf55 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:06.589538 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 391/802] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jun 16 15:59:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:06.590948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00433e3e-cde2-4d7d-8af3-74b83862bdf9 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:06.591230 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00433e3e-cde2-4d7d-8af3-74b83862bdf9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:06.591510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00433e3e-cde2-4d7d-8af3-74b83862bdf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:06.610816 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-00433e3e-cde2-4d7d-8af3-74b83862bdf9 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:06.612039 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-00433e3e-cde2-4d7d-8af3-74b83862bdf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:06.621744 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8239ff-a913-411d-b7d8-d8627574b709 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:06.678376 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65814b-803a-4976-8352-3f51dd87fbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:06.679355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65814b-803a-4976-8352-3f51dd87fbfd None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:06.680340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65814b-803a-4976-8352-3f51dd87fbfd None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:06.680791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65814b-803a-4976-8352-3f51dd87fbfd None admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:06.682853 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65814b-803a-4976-8352-3f51dd87fbfd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:06.683339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65814b-803a-4976-8352-3f51dd87fbfd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:06.686676 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65814b-803a-4976-8352-3f51dd87fbfd None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:06.700404 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:06.705112 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 412/803] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:06.705919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65814b-803a-4976-8352-3f51dd87fbfd None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:06.723918 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 392/804] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:06.726193 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00433e3e-cde2-4d7d-8af3-74b83862bdf9 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:06.851294 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65814b-803a-4976-8352-3f51dd87fbfd None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:06.863881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:06.873901 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00433e3e-cde2-4d7d-8af3-74b83862bdf9 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:06.891742 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8edfe-6a53-49e4-a768-35f8be35f03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:06.892704 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8edfe-6a53-49e4-a768-35f8be35f03e None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:06.893100 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8edfe-6a53-49e4-a768-35f8be35f03e None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:06.893459 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8edfe-6a53-49e4-a768-35f8be35f03e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:06.902319 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8edfe-6a53-49e4-a768-35f8be35f03e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:06.903793 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8edfe-6a53-49e4-a768-35f8be35f03e None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:06.915270 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf32419-bcef-45de-9498-99e17578884d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:06.916245 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf32419-bcef-45de-9498-99e17578884d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:06.916657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf32419-bcef-45de-9498-99e17578884d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:06.917160 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf32419-bcef-45de-9498-99e17578884d None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:06.918192 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf32419-bcef-45de-9498-99e17578884d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:06.919399 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf32419-bcef-45de-9498-99e17578884d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:06.919530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-43d8edfe-6a53-49e4-a768-35f8be35f03e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 15:59:06.919530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:06.919530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:06.919530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:06.919530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:06.919530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:06.919530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:06.919530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:06.919530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:06.919530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:06.919530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:06.919530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:06.919530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:06.919530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:06.919530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:06.919530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:06.921548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:06.921548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:06.921548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:06.921548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:06.921548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:06.921548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:06.921548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:06.921548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:06.921548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:06.921548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:06.921548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:06.921548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:06.921548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:06.921548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:06.921548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:06.921548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:06.921548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:06.921548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:06.923517 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:06.923517 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:06.923517 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:06.923517 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:06.923517 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:06.923517 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:06.923517 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:06.923517 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:06.923517 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 15:59:06.923517 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:06.923517 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 413/805] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jun 16 15:59:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:06.924455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf32419-bcef-45de-9498-99e17578884d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:06.924547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8edfe-6a53-49e4-a768-35f8be35f03e None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:06.930878 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 393/806] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jun 16 15:59:06 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:06.932169 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf32419-bcef-45de-9498-99e17578884d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:07.007364 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:07.049101 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8edfe-6a53-49e4-a768-35f8be35f03e None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:07.064656 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf32419-bcef-45de-9498-99e17578884d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:07.089786 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bcd3bd-e01d-4c76-9cdf-0345f35b7b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:07.090712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bcd3bd-e01d-4c76-9cdf-0345f35b7b6b None admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:07.090997 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bcd3bd-e01d-4c76-9cdf-0345f35b7b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:07.091278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bcd3bd-e01d-4c76-9cdf-0345f35b7b6b None admin] PATH_INFO: `/v3/projects/b1177a85a0de4276898daf4a24ff069f/users/0082e37ce54f49f7837601349508c89f/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:07.094020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bcd3bd-e01d-4c76-9cdf-0345f35b7b6b None admin] RBAC: Authorizing `identity:create_grant(project_id=b1177a85a0de4276898daf4a24ff069f, user_id=0082e37ce54f49f7837601349508c89f, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:07.095500 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bcd3bd-e01d-4c76-9cdf-0345f35b7b6b None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:07.106221 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a177c74-3b7f-4d71-95c2-bc7d4035652e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:07.107030 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a177c74-3b7f-4d71-95c2-bc7d4035652e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:07.107308 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a177c74-3b7f-4d71-95c2-bc7d4035652e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:07.107725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a177c74-3b7f-4d71-95c2-bc7d4035652e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:07.110495 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 414/807] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 15:59:06 2022] PUT /identity/v3/projects/b1177a85a0de4276898daf4a24ff069f/users/0082e37ce54f49f7837601349508c89f/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:59:07.111883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c373a-65a7-4a86-afd9-96d303faa178 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:07.112173 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c373a-65a7-4a86-afd9-96d303faa178 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:07.112454 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c373a-65a7-4a86-afd9-96d303faa178 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:07.118048 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 415/808] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:07.118048 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c373a-65a7-4a86-afd9-96d303faa178 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:07.129762 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a177c74-3b7f-4d71-95c2-bc7d4035652e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:07.138794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-3a177c74-3b7f-4d71-95c2-bc7d4035652e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:59:07.138794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:07.138794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:07.138794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:07.138794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:07.138794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:07.138794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:07.138794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:07.138794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:07.138794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:07.138794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:07.138794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:07.138794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:07.138794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:07.138794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:07.138794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:07.140516 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:07.140516 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:07.140516 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:07.140516 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:07.140516 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:07.140516 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:07.140516 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:07.140516 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:07.140516 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:07.140516 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:07.140516 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:07.140516 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:07.140516 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:07.140516 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:07.140516 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:07.140516 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:07.142900 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:07.142900 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:07.142900 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:07.142900 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:07.142900 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:07.142900 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:07.142900 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:07.142900 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:07.142900 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:07.142900 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:07.142900 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:59:07.142900 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:07.142900 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 394/809] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 15:59:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:07.142900 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a177c74-3b7f-4d71-95c2-bc7d4035652e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:07.169581 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:07.268989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c373a-65a7-4a86-afd9-96d303faa178 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:07.274723 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a177c74-3b7f-4d71-95c2-bc7d4035652e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:07.303616 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:07.328368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d531258a-5bb1-4729-b36a-4e77d717af68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:07.328604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5bd265-89cd-4f4a-920f-b527fdd0da15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:07.329108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531258a-5bb1-4729-b36a-4e77d717af68 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:07.329391 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531258a-5bb1-4729-b36a-4e77d717af68 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:07.329656 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bd265-89cd-4f4a-920f-b527fdd0da15 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:07.329769 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531258a-5bb1-4729-b36a-4e77d717af68 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:07.330148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bd265-89cd-4f4a-920f-b527fdd0da15 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:07.330467 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d531258a-5bb1-4729-b36a-4e77d717af68 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:07.330582 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bd265-89cd-4f4a-920f-b527fdd0da15 None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:07.331544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5bd265-89cd-4f4a-920f-b527fdd0da15 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:07.331899 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d531258a-5bb1-4729-b36a-4e77d717af68 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:07.332049 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5bd265-89cd-4f4a-920f-b527fdd0da15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:07.333649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5bd265-89cd-4f4a-920f-b527fdd0da15 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:07.358571 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 416/810] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jun 16 15:59:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:07.360173 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f7ec9-e630-4a65-aa11-6f430f105afc None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:07.360487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f7ec9-e630-4a65-aa11-6f430f105afc None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:07.360746 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f7ec9-e630-4a65-aa11-6f430f105afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:07.363396 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 395/811] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jun 16 15:59:07 2022] POST /identity/v3/endpoints => generated 346 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:59:07.364681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d531258a-5bb1-4729-b36a-4e77d717af68 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:07.381474 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-352f7ec9-e630-4a65-aa11-6f430f105afc None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:07.382796 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-352f7ec9-e630-4a65-aa11-6f430f105afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:07.492214 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 417/812] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:07.493791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f7ec9-e630-4a65-aa11-6f430f105afc None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:07.523149 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d531258a-5bb1-4729-b36a-4e77d717af68 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:07.527461 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:07.575262 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30077704-6c67-47c1-b254-966bb7ad00f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:07.575745 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30077704-6c67-47c1-b254-966bb7ad00f8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:07.576038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30077704-6c67-47c1-b254-966bb7ad00f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:07.576348 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30077704-6c67-47c1-b254-966bb7ad00f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:07.577188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30077704-6c67-47c1-b254-966bb7ad00f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:07.577516 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30077704-6c67-47c1-b254-966bb7ad00f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:07.579091 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30077704-6c67-47c1-b254-966bb7ad00f8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:07.589347 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 396/813] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:07.590716 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30077704-6c67-47c1-b254-966bb7ad00f8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:07.637911 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:07.658992 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f7ec9-e630-4a65-aa11-6f430f105afc None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:07.713852 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95a1c3-da85-43f9-a1a5-b66360042fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:07.714705 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95a1c3-da85-43f9-a1a5-b66360042fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:07.715053 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95a1c3-da85-43f9-a1a5-b66360042fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:07.715251 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95a1c3-da85-43f9-a1a5-b66360042fb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:07.732061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc95a1c3-da85-43f9-a1a5-b66360042fb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:07.735383 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc95a1c3-da85-43f9-a1a5-b66360042fb3 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:07.741854 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30077704-6c67-47c1-b254-966bb7ad00f8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:07.763446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-fc95a1c3-da85-43f9-a1a5-b66360042fb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:07.763446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:07.763446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:07.763446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:07.763446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:07.763446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:07.763446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:07.763446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:07.763446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:07.763446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:07.763446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:07.763446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:07.763446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:07.763446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:07.763446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:07.764738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:07.764738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:07.764738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:07.764738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:07.764738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:07.764738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:07.764738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:07.764738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:07.764738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:07.764738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:07.764738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:07.764738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:07.764738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:07.764738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:07.764738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:07.764738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:07.764738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:07.766103 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:07.766103 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:07.766103 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:07.766103 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:07.766103 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 418/814] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:07.766103 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95a1c3-da85-43f9-a1a5-b66360042fb3 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:07.801611 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb414a-201e-4dab-b0b1-c845f81cf0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:07.802735 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb414a-201e-4dab-b0b1-c845f81cf0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:07.803176 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb414a-201e-4dab-b0b1-c845f81cf0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:07.803654 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb414a-201e-4dab-b0b1-c845f81cf0d2 None admin] PATH_INFO: `/v3/services/network` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:07.804498 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb414a-201e-4dab-b0b1-c845f81cf0d2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:07.806160 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb414a-201e-4dab-b0b1-c845f81cf0d2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:07.819458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-40bb414a-201e-4dab-b0b1-c845f81cf0d2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 15:59:07.819458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:07.819458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 15:59:07.819458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 15:59:07.819458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 15:59:07.819458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 15:59:07.819458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 15:59:07.819458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:07.819458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 15:59:07.819458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:07.819458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:59:07.819458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:07.819458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:07.819458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:07.819458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:07.819458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:07.819458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:07.820714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:07.820714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:07.820714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:07.820714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:07.820714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:07.820714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:07.820714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 15:59:07.820714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 15:59:07.820714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 15:59:07.820714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 15:59:07.820714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:07.820714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:07.820714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:07.820714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:07.820714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:07.820714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:07.821681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:07.821681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:07.821681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:07.821681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:07.821681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:07.821681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:07.821681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:07.821681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:07.821681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:07.821681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:07.821681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 15:59:07.821681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:07.821681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 15:59:07.821681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:07.824393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 397/815] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:07 2022] GET /identity/v3/services/network => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:07.825614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb414a-201e-4dab-b0b1-c845f81cf0d2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:07.877359 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:07.919271 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95a1c3-da85-43f9-a1a5-b66360042fb3 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:07.952692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb414a-201e-4dab-b0b1-c845f81cf0d2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:07.957309 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:07.961672 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54710068-1cc6-4853-aa74-c2a44a15391a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:07.963378 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54710068-1cc6-4853-aa74-c2a44a15391a None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:07.963793 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54710068-1cc6-4853-aa74-c2a44a15391a None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:07.964176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54710068-1cc6-4853-aa74-c2a44a15391a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:07.977640 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54710068-1cc6-4853-aa74-c2a44a15391a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:07.979476 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54710068-1cc6-4853-aa74-c2a44a15391a None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:07.994698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf787f2-4161-4198-b48b-7eb10d0809ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:07.995630 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf787f2-4161-4198-b48b-7eb10d0809ce None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:07.996099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf787f2-4161-4198-b48b-7eb10d0809ce None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:07.996528 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf787f2-4161-4198-b48b-7eb10d0809ce None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:07.997484 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf787f2-4161-4198-b48b-7eb10d0809ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:07.998017 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf787f2-4161-4198-b48b-7eb10d0809ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:08.000316 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf787f2-4161-4198-b48b-7eb10d0809ce None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:08.002652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-54710068-1cc6-4853-aa74-c2a44a15391a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:08.002652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:08.002652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:08.002652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:08.002652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:08.002652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:08.002652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:08.002652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:08.002652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:08.002652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:08.002652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:08.002652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:08.002652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:08.002652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:08.002652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:08.003529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:08.003529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:08.003529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:08.003529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:08.003529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:08.003529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:08.003529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:08.003529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:08.003529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:08.003529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:08.003529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:08.003529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:08.003529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:08.003529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:08.003529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:08.003529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:08.003529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:08.004422 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:08.004422 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:08.004422 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:08.004422 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:08.004422 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 419/816] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:08.006357 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54710068-1cc6-4853-aa74-c2a44a15391a None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:08.018301 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 398/817] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:08.019424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf787f2-4161-4198-b48b-7eb10d0809ce None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:08.130510 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54710068-1cc6-4853-aa74-c2a44a15391a None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:08.132161 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf787f2-4161-4198-b48b-7eb10d0809ce None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:08.152535 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:08.173177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340103e-5937-4946-9161-fbaedd9a1606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:08.174245 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340103e-5937-4946-9161-fbaedd9a1606 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:08.174698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340103e-5937-4946-9161-fbaedd9a1606 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:08.175120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340103e-5937-4946-9161-fbaedd9a1606 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:08.175487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc8e48-954a-4695-b45a-f97340a4fc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:08.176278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc8e48-954a-4695-b45a-f97340a4fc2e None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:08.176663 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc8e48-954a-4695-b45a-f97340a4fc2e None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:08.176839 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9340103e-5937-4946-9161-fbaedd9a1606 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:08.177125 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc8e48-954a-4695-b45a-f97340a4fc2e None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:08.177429 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9340103e-5937-4946-9161-fbaedd9a1606 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:08.178082 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc8e48-954a-4695-b45a-f97340a4fc2e None admin] RBAC: Adding query filter params (name=network) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:08.178495 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc8e48-954a-4695-b45a-f97340a4fc2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:08.179316 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9340103e-5937-4946-9161-fbaedd9a1606 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:08.180075 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc8e48-954a-4695-b45a-f97340a4fc2e None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:08.193364 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 420/818] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jun 16 15:59:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:08.194456 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 399/819] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:08.194806 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc8e48-954a-4695-b45a-f97340a4fc2e None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:08.198770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340103e-5937-4946-9161-fbaedd9a1606 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:08.224996 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:08.318159 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340103e-5937-4946-9161-fbaedd9a1606 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:08.340485 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc8e48-954a-4695-b45a-f97340a4fc2e None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:08.358509 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649cd791-156a-4859-b02f-7845e6f52a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:08.359684 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cd791-156a-4859-b02f-7845e6f52a0e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:08.360239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cd791-156a-4859-b02f-7845e6f52a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:08.360600 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cd791-156a-4859-b02f-7845e6f52a0e None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:08.361512 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649cd791-156a-4859-b02f-7845e6f52a0e None admin] RBAC: Adding query filter params (type=network) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:08.361851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649cd791-156a-4859-b02f-7845e6f52a0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:08.363406 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649cd791-156a-4859-b02f-7845e6f52a0e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:08.373962 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 400/820] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jun 16 15:59:08 2022] GET /identity/v3/services?type=network => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:08.375504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649cd791-156a-4859-b02f-7845e6f52a0e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:08.410568 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa031f2-04e0-4bf3-a10f-e7f4e3864acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:08.411980 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa031f2-04e0-4bf3-a10f-e7f4e3864acc None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:08.412423 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa031f2-04e0-4bf3-a10f-e7f4e3864acc None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:08.412987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa031f2-04e0-4bf3-a10f-e7f4e3864acc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:08.427943 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa031f2-04e0-4bf3-a10f-e7f4e3864acc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:08.430663 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa031f2-04e0-4bf3-a10f-e7f4e3864acc None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:08.460114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-2fa031f2-04e0-4bf3-a10f-e7f4e3864acc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:08.460114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:08.460114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:08.460114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:08.460114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:08.460114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:08.460114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:08.460114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:08.460114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:08.460114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:08.460114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:08.460114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:08.460114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:08.460114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:08.460114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:08.462190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:08.462190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:08.462190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:08.462190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:08.462190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:08.462190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:08.462190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:08.462190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:08.462190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:08.462190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:08.462190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:08.462190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:08.462190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:08.462190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:08.462190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:08.462190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:08.462190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:08.463966 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:08.463966 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:08.463966 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:08.463966 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:08.463966 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:08.463966 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:08.463966 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:08.463966 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:08.463966 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 421/821] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:08.463966 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa031f2-04e0-4bf3-a10f-e7f4e3864acc None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:08.544228 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:08.577550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:08.625151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649cd791-156a-4859-b02f-7845e6f52a0e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:08.643859 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa031f2-04e0-4bf3-a10f-e7f4e3864acc None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:08.679195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87196e-00cc-47d9-a8bf-902110feb86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:08.680728 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87196e-00cc-47d9-a8bf-902110feb86a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:08.681244 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87196e-00cc-47d9-a8bf-902110feb86a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:08.681872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87196e-00cc-47d9-a8bf-902110feb86a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:08.689528 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43f5a7-834b-4f5a-9029-621613687bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:08.690329 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43f5a7-834b-4f5a-9029-621613687bcb None admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:08.690751 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43f5a7-834b-4f5a-9029-621613687bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:08.691196 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43f5a7-834b-4f5a-9029-621613687bcb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:08.692350 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab43f5a7-834b-4f5a-9029-621613687bcb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:08.694621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab43f5a7-834b-4f5a-9029-621613687bcb None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:08.697802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87196e-00cc-47d9-a8bf-902110feb86a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:08.702089 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87196e-00cc-47d9-a8bf-902110feb86a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:08.727793 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 422/822] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jun 16 15:59:08 2022] POST /identity/v3/endpoints => generated 327 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:59:08.728768 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43f5a7-834b-4f5a-9029-621613687bcb None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:08.739625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-8f87196e-00cc-47d9-a8bf-902110feb86a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 15:59:08.739625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:08.739625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:08.739625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:08.739625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:08.739625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:08.739625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:08.739625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:08.739625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:08.739625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:08.739625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:08.739625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:08.739625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:08.739625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:08.739625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:08.739625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:08.741514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:08.741514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:08.741514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:08.741514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:08.741514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:08.741514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:08.741514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:08.741514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:08.741514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:08.741514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:08.741514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:08.741514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:08.741514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:08.741514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:08.741514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:08.741514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:08.741514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:08.741514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:08.743856 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:08.743856 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:08.743856 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:08.743856 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:08.743856 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:08.743856 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:08.743856 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:08.743856 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:08.743856 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 15:59:08.743856 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:08.750207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 401/823] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jun 16 15:59:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:08.755188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87196e-00cc-47d9-a8bf-902110feb86a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:08.951221 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:08.954059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87196e-00cc-47d9-a8bf-902110feb86a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:08.968727 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43f5a7-834b-4f5a-9029-621613687bcb None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:09.000847 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8e716-f7a9-4f9d-817a-1b0350b0a962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:09.001880 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8e716-f7a9-4f9d-817a-1b0350b0a962 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:09.002291 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8e716-f7a9-4f9d-817a-1b0350b0a962 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:09.002665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8e716-f7a9-4f9d-817a-1b0350b0a962 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:09.004152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e8e716-f7a9-4f9d-817a-1b0350b0a962 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:09.004711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e8e716-f7a9-4f9d-817a-1b0350b0a962 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:09.006459 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e8e716-f7a9-4f9d-817a-1b0350b0a962 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:09.023942 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b36d21-cf4f-4153-9d57-327cea262bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:09.025112 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b36d21-cf4f-4153-9d57-327cea262bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:09.025562 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b36d21-cf4f-4153-9d57-327cea262bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:09.025844 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b36d21-cf4f-4153-9d57-327cea262bc5 None admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:09.026690 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b36d21-cf4f-4153-9d57-327cea262bc5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:09.026996 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b36d21-cf4f-4153-9d57-327cea262bc5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:09.028612 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b36d21-cf4f-4153-9d57-327cea262bc5 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:09.041618 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 402/824] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jun 16 15:59:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:09.047590 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8e716-f7a9-4f9d-817a-1b0350b0a962 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:09.048865 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 423/825] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:09.050945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:09.055090 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b36d21-cf4f-4153-9d57-327cea262bc5 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:09.196251 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8e716-f7a9-4f9d-817a-1b0350b0a962 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:09.225178 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b36d21-cf4f-4153-9d57-327cea262bc5 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:09.245905 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d60aae-681e-46c4-9f0c-544ed64cbc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:09.247156 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d60aae-681e-46c4-9f0c-544ed64cbc22 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:09.247615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d60aae-681e-46c4-9f0c-544ed64cbc22 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:09.248868 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d60aae-681e-46c4-9f0c-544ed64cbc22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:09.271434 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa4e86-4661-4356-93fb-f4dd000a7451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:09.272767 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa4e86-4661-4356-93fb-f4dd000a7451 None admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:09.273251 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa4e86-4661-4356-93fb-f4dd000a7451 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:09.273707 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa4e86-4661-4356-93fb-f4dd000a7451 None admin] PATH_INFO: `/v3/projects/b1177a85a0de4276898daf4a24ff069f/users/2c2e0e19880f4b048faf034dd38c7365/roles/cfc67444e9944ffc87173be8c9245108` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:09.302562 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aa4e86-4661-4356-93fb-f4dd000a7451 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1177a85a0de4276898daf4a24ff069f, user_id=2c2e0e19880f4b048faf034dd38c7365, role_id=cfc67444e9944ffc87173be8c9245108)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:09.304590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aa4e86-4661-4356-93fb-f4dd000a7451 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:09.322618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 424/826] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 15:59:09 2022] PUT /identity/v3/projects/b1177a85a0de4276898daf4a24ff069f/users/2c2e0e19880f4b048faf034dd38c7365/roles/cfc67444e9944ffc87173be8c9245108 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:59:09.363696 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d60aae-681e-46c4-9f0c-544ed64cbc22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:09.365611 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d60aae-681e-46c4-9f0c-544ed64cbc22 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:09.368319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:09.497017 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 403/827] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:09 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 15:59:09.506658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa4e86-4661-4356-93fb-f4dd000a7451 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:09.514496 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:09.651954 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa4e86-4661-4356-93fb-f4dd000a7451 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:09.654539 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814db1c5-2691-4fbe-b080-2f2c158c373c None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:09.655350 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814db1c5-2691-4fbe-b080-2f2c158c373c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:09.655633 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814db1c5-2691-4fbe-b080-2f2c158c373c None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:09.657055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 404/828] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:09.663224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771df2fa-57c7-4aec-b723-98c9d96cb9c1 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:09.663652 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771df2fa-57c7-4aec-b723-98c9d96cb9c1 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:09.664029 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771df2fa-57c7-4aec-b723-98c9d96cb9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:09.682003 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-771df2fa-57c7-4aec-b723-98c9d96cb9c1 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:09.683352 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-771df2fa-57c7-4aec-b723-98c9d96cb9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:09.698385 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad928145-5bc4-45f7-a395-64333687ac8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:09.699771 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad928145-5bc4-45f7-a395-64333687ac8d None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:09.700308 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad928145-5bc4-45f7-a395-64333687ac8d None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:09.700836 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad928145-5bc4-45f7-a395-64333687ac8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:09.715696 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad928145-5bc4-45f7-a395-64333687ac8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:09.719088 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad928145-5bc4-45f7-a395-64333687ac8d None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:09.745322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-ad928145-5bc4-45f7-a395-64333687ac8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:09.745322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:09.745322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:09.745322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:09.745322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:09.745322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:09.745322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:09.745322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:09.745322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:09.745322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:09.745322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:09.745322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:09.745322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:09.745322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:09.745322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:09.746409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:09.746409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:09.746409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:09.746409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:09.746409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:09.746409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:09.746409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:09.746409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:09.746409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:09.746409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:09.746409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:09.746409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:09.746409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:09.746409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:09.746409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:09.746409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:09.746409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:09.747462 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:09.747462 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:09.747462 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:09.747462 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:09.747721 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 425/829] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:09.754035 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad928145-5bc4-45f7-a395-64333687ac8d None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:09.776756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 405/830] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:09 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 15:59:09.805132 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771df2fa-57c7-4aec-b723-98c9d96cb9c1 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:09.841740 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:09.930192 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771df2fa-57c7-4aec-b723-98c9d96cb9c1 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:09.972384 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586f7e6-bc12-4e01-a35a-1c0e6092c103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:09.973203 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586f7e6-bc12-4e01-a35a-1c0e6092c103 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:09.973489 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586f7e6-bc12-4e01-a35a-1c0e6092c103 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:09.973768 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586f7e6-bc12-4e01-a35a-1c0e6092c103 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:09.985963 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586f7e6-bc12-4e01-a35a-1c0e6092c103 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:09.994093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-6586f7e6-bc12-4e01-a35a-1c0e6092c103 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:59:09.994093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:09.994093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:09.994093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:09.994093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:09.994093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:09.994093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:09.994093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:09.994093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:09.994093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:09.994093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:09.994093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:09.994093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:09.994093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:09.994093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:09.994093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:09.995099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:09.995099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:09.995099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:09.995099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:09.995099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:09.995099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:09.995099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:09.995099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:09.995099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:09.995099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:09.995099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:09.995099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:09.995099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:09.995099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:09.995099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:09.995099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:09.996430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:09.996430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:09.996430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:09.996430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:09.996430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:09.996430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:09.996430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:09.996430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:09.996430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:09.996430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:09.996430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:59:09.996430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:09.996430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 406/831] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 15:59:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:09.997451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d686b8-8da1-436b-bbc0-a7258e5d7f34 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:09.997451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d686b8-8da1-436b-bbc0-a7258e5d7f34 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:09.997451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d686b8-8da1-436b-bbc0-a7258e5d7f34 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:09.999541 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 407/832] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:10.005000 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68785c35-3801-41b2-ba5d-3c85242ade79 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:10.005351 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68785c35-3801-41b2-ba5d-3c85242ade79 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:10.005569 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68785c35-3801-41b2-ba5d-3c85242ade79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:10.014169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:10.024909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-68785c35-3801-41b2-ba5d-3c85242ade79 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:10.026146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-68785c35-3801-41b2-ba5d-3c85242ade79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:10.067930 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad928145-5bc4-45f7-a395-64333687ac8d None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:10.108903 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa75f4-8b93-4f55-b395-a2bac72ab556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:10.109592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa75f4-8b93-4f55-b395-a2bac72ab556 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:10.109866 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa75f4-8b93-4f55-b395-a2bac72ab556 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:10.110277 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 408/833] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:10 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:10.111168 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68785c35-3801-41b2-ba5d-3c85242ade79 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:10.114220 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa75f4-8b93-4f55-b395-a2bac72ab556 None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:10.116273 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa75f4-8b93-4f55-b395-a2bac72ab556 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:10.116835 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa75f4-8b93-4f55-b395-a2bac72ab556 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:10.118486 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa75f4-8b93-4f55-b395-a2bac72ab556 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:10.135555 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 426/834] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:10.137216 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa75f4-8b93-4f55-b395-a2bac72ab556 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:10.141082 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:10.232729 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68785c35-3801-41b2-ba5d-3c85242ade79 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:10.259097 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa75f4-8b93-4f55-b395-a2bac72ab556 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:10.274482 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3a602-5462-4def-a517-a0f79dbf692c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:10.275180 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3a602-5462-4def-a517-a0f79dbf692c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:10.275455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3a602-5462-4def-a517-a0f79dbf692c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:10.275727 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3a602-5462-4def-a517-a0f79dbf692c None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:10.276484 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3a602-5462-4def-a517-a0f79dbf692c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:10.276787 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3a602-5462-4def-a517-a0f79dbf692c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:10.278233 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3a602-5462-4def-a517-a0f79dbf692c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:10.283563 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:10.297977 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 409/835] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:10.297977 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3a602-5462-4def-a517-a0f79dbf692c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:10.316743 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7ee16-5053-432c-9789-083c2af4ca00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:10.317689 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7ee16-5053-432c-9789-083c2af4ca00 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:10.318184 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7ee16-5053-432c-9789-083c2af4ca00 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:10.318636 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7ee16-5053-432c-9789-083c2af4ca00 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:10.319750 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb7ee16-5053-432c-9789-083c2af4ca00 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:10.322161 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb7ee16-5053-432c-9789-083c2af4ca00 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:10.344253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-9fb7ee16-5053-432c-9789-083c2af4ca00 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 15:59:10.344253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:10.344253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 15:59:10.344253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 15:59:10.344253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 15:59:10.344253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 15:59:10.344253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 15:59:10.344253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:10.344253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 15:59:10.344253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:10.344253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:59:10.344253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:10.344253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:10.344253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:10.344253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:10.344253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:10.344253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:10.345500 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:10.345500 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:10.345500 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:10.345500 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:10.345500 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:10.345500 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:10.345500 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 15:59:10.345500 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 15:59:10.345500 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 15:59:10.345500 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 15:59:10.345500 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:10.345500 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:10.345500 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:10.345500 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:10.345500 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:10.345500 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:10.346416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:10.346416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:10.346416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:10.346416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:10.346416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:10.346416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:10.346416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:10.346416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:10.346416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:10.346416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:10.346416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 15:59:10.346416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:10.346416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 15:59:10.346416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:10.346416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 427/836] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 15:59:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:10.347225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7ee16-5053-432c-9789-083c2af4ca00 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:10.432936 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:10.456955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3a602-5462-4def-a517-a0f79dbf692c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:10.488956 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7ee16-5053-432c-9789-083c2af4ca00 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:10.499186 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f8203a-fe8f-4da6-ad9c-66ead9623e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:10.500059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8203a-fe8f-4da6-ad9c-66ead9623e29 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:10.500347 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8203a-fe8f-4da6-ad9c-66ead9623e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:10.500614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8203a-fe8f-4da6-ad9c-66ead9623e29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:10.508553 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f8203a-fe8f-4da6-ad9c-66ead9623e29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:10.510099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f8203a-fe8f-4da6-ad9c-66ead9623e29 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:10.525434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-60f8203a-fe8f-4da6-ad9c-66ead9623e29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:10.525434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:10.525434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:10.525434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:10.525434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:10.525434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:10.525434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:10.525434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:10.525434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:10.525434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:10.525434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:10.525434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:10.525434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:10.525434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:10.525434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:10.526576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:10.526576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:10.526576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:10.526576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:10.526576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:10.526576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:10.526576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:10.526576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:10.526576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:10.526576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:10.526576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:10.526576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:10.526576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:10.526576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:10.526576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:10.526576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:10.526576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:10.527479 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:10.527479 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:10.527479 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:10.527479 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:10.527479 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:10.527479 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:10.527479 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:10.527479 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:10.530307 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 410/837] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:10.531507 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f8203a-fe8f-4da6-ad9c-66ead9623e29 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:10.549096 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a666cfbe-ee46-4b03-b5e2-90a120099fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:10.550061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666cfbe-ee46-4b03-b5e2-90a120099fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:10.550548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666cfbe-ee46-4b03-b5e2-90a120099fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:10.551004 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666cfbe-ee46-4b03-b5e2-90a120099fb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:10.565432 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a666cfbe-ee46-4b03-b5e2-90a120099fb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:10.567306 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a666cfbe-ee46-4b03-b5e2-90a120099fb8 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:10.588790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-a666cfbe-ee46-4b03-b5e2-90a120099fb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:10.588790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:10.588790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:10.588790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:10.588790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:10.588790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:10.588790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:10.588790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:10.588790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:10.588790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:10.588790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:10.588790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:10.588790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:10.588790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:10.588790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:10.589774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:10.589774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:10.589774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:10.589774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:10.589774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:10.589774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:10.589774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:10.589774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:10.589774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:10.589774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:10.589774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:10.589774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:10.589774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:10.589774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:10.589774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:10.589774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:10.589774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:10.596223 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:10.596223 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:10.596223 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:10.596223 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:10.596223 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 428/838] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:10.596223 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a666cfbe-ee46-4b03-b5e2-90a120099fb8 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:10.655101 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:10.675298 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f8203a-fe8f-4da6-ad9c-66ead9623e29 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:10.725319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8cbf2-3ca9-4f02-82e5-1b8a95fb07ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:10.725873 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8cbf2-3ca9-4f02-82e5-1b8a95fb07ad None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:10.726209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8cbf2-3ca9-4f02-82e5-1b8a95fb07ad None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:10.726860 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8cbf2-3ca9-4f02-82e5-1b8a95fb07ad None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:10.727203 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8cbf2-3ca9-4f02-82e5-1b8a95fb07ad None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:10.727514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8cbf2-3ca9-4f02-82e5-1b8a95fb07ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:10.729294 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8cbf2-3ca9-4f02-82e5-1b8a95fb07ad None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:10.742870 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 411/839] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 15:59:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:10.742870 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8cbf2-3ca9-4f02-82e5-1b8a95fb07ad None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:10.753663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:10.766237 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a666cfbe-ee46-4b03-b5e2-90a120099fb8 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:10.820305 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a244ddb-a0a2-4e1f-b9d2-24b41e4ef423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:10.821196 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a244ddb-a0a2-4e1f-b9d2-24b41e4ef423 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:10.821488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a244ddb-a0a2-4e1f-b9d2-24b41e4ef423 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:10.821766 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a244ddb-a0a2-4e1f-b9d2-24b41e4ef423 None admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:10.823019 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a244ddb-a0a2-4e1f-b9d2-24b41e4ef423 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:10.823326 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a244ddb-a0a2-4e1f-b9d2-24b41e4ef423 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:10.824830 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a244ddb-a0a2-4e1f-b9d2-24b41e4ef423 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:10.847933 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 429/840] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:10.847933 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a244ddb-a0a2-4e1f-b9d2-24b41e4ef423 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:10.907455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8cbf2-3ca9-4f02-82e5-1b8a95fb07ad None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:10.951172 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f3570-3aeb-49cd-8649-7830084368e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:10.952796 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f3570-3aeb-49cd-8649-7830084368e2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:10.953228 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f3570-3aeb-49cd-8649-7830084368e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:10.953653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f3570-3aeb-49cd-8649-7830084368e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:10.954797 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f3570-3aeb-49cd-8649-7830084368e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:10.955420 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f3570-3aeb-49cd-8649-7830084368e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:10.957318 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f3570-3aeb-49cd-8649-7830084368e2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:10.977668 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 412/841] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:10.977668 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f3570-3aeb-49cd-8649-7830084368e2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:11.002294 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:11.020935 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a244ddb-a0a2-4e1f-b9d2-24b41e4ef423 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:11.064426 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:11.077151 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c44102-67c4-4334-8772-b6d29444aa51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:11.077843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c44102-67c4-4334-8772-b6d29444aa51 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:11.078148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c44102-67c4-4334-8772-b6d29444aa51 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:11.078423 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c44102-67c4-4334-8772-b6d29444aa51 None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:11.079163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c44102-67c4-4334-8772-b6d29444aa51 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:11.079469 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c44102-67c4-4334-8772-b6d29444aa51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:11.081051 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c44102-67c4-4334-8772-b6d29444aa51 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:11.087820 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 430/842] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 15:59:10 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:11.088953 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c44102-67c4-4334-8772-b6d29444aa51 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:11.137776 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f3570-3aeb-49cd-8649-7830084368e2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:11.208683 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676bcf0-bffc-4f29-b8cf-ee8c88501f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:11.209963 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676bcf0-bffc-4f29-b8cf-ee8c88501f7d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:11.210360 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676bcf0-bffc-4f29-b8cf-ee8c88501f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:11.210740 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676bcf0-bffc-4f29-b8cf-ee8c88501f7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:11.212379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676bcf0-bffc-4f29-b8cf-ee8c88501f7d None admin] RBAC: Adding query filter params (role.id=bbbc77ec645d4ff5a8bf29ce0377f578, scope.project.id=b1177a85a0de4276898daf4a24ff069f, user.id=c3645b1a0be34c01ac87c734fcd85ff6) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:11.212811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676bcf0-bffc-4f29-b8cf-ee8c88501f7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:11.214657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676bcf0-bffc-4f29-b8cf-ee8c88501f7d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:11.230235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 413/843] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:59:10 2022] GET /identity/v3/role_assignments?user.id=c3645b1a0be34c01ac87c734fcd85ff6&scope.project.id=b1177a85a0de4276898daf4a24ff069f&role.id=bbbc77ec645d4ff5a8bf29ce0377f578 => generated 608 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:11.298111 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c44102-67c4-4334-8772-b6d29444aa51 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:11.356054 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23dde8f-c1f1-4cdc-ace1-a6eca49656c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:11.357139 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23dde8f-c1f1-4cdc-ace1-a6eca49656c7 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:11.357579 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23dde8f-c1f1-4cdc-ace1-a6eca49656c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:11.358403 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23dde8f-c1f1-4cdc-ace1-a6eca49656c7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:11.370198 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23dde8f-c1f1-4cdc-ace1-a6eca49656c7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:11.372213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23dde8f-c1f1-4cdc-ace1-a6eca49656c7 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:11.395370 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-a23dde8f-c1f1-4cdc-ace1-a6eca49656c7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 15:59:11.395370 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:11.395370 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:11.395370 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:11.395370 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:11.395370 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:11.395370 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:11.395370 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:11.395370 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:11.395370 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:11.395370 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:11.395370 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:11.395370 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:11.395370 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:11.395370 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:11.395370 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:11.397591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:11.397591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:11.397591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:11.397591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:11.397591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:11.397591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:11.397591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:11.397591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:11.397591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:11.397591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:11.397591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:11.397591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:11.397591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:11.397591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:11.397591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:11.397591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:11.397591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:11.397591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:11.399878 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:11.399878 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:11.399878 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:11.399878 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:11.399878 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:11.399878 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:11.399878 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:11.399878 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:11.399878 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 15:59:11.399878 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:11.399878 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 431/844] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 15:59:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:11.406070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676bcf0-bffc-4f29-b8cf-ee8c88501f7d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:11.549491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676bcf0-bffc-4f29-b8cf-ee8c88501f7d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:11.587494 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:11.595838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d240d-fcc1-4930-badf-f0da139ef19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:11.596828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d240d-fcc1-4930-badf-f0da139ef19a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:11.597126 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d240d-fcc1-4930-badf-f0da139ef19a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:11.597404 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d240d-fcc1-4930-badf-f0da139ef19a None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:11.598317 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d240d-fcc1-4930-badf-f0da139ef19a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:11.598634 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d240d-fcc1-4930-badf-f0da139ef19a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:11.600116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d240d-fcc1-4930-badf-f0da139ef19a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:11.630747 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 414/845] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jun 16 15:59:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:11.636811 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23dde8f-c1f1-4cdc-ace1-a6eca49656c7 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:11.674635 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:11.710675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ebb4b-46b5-4a09-a3ae-b9e41c5b9427 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:11.711240 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ebb4b-46b5-4a09-a3ae-b9e41c5b9427 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:11.711708 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ebb4b-46b5-4a09-a3ae-b9e41c5b9427 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:11.714682 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 415/846] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:11.721838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c9d90-48ee-429a-84bc-a1a01fb0918e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:11.723204 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c9d90-48ee-429a-84bc-a1a01fb0918e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:11.723644 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c9d90-48ee-429a-84bc-a1a01fb0918e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:11.767343 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4d7c9d90-48ee-429a-84bc-a1a01fb0918e None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:11.769262 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4d7c9d90-48ee-429a-84bc-a1a01fb0918e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:11.829619 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23dde8f-c1f1-4cdc-ace1-a6eca49656c7 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:11.893893 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 416/847] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:11 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 15:59:11.904745 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd7e40-2847-4994-98e8-df9589bd537b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:11.906125 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd7e40-2847-4994-98e8-df9589bd537b None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:11.906690 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd7e40-2847-4994-98e8-df9589bd537b None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:11.907168 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd7e40-2847-4994-98e8-df9589bd537b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:11.939146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c9d90-48ee-429a-84bc-a1a01fb0918e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:12.059187 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fd7e40-2847-4994-98e8-df9589bd537b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:12.064962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fd7e40-2847-4994-98e8-df9589bd537b None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:12.114091 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:12.137533 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c9d90-48ee-429a-84bc-a1a01fb0918e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:12.176380 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:12.203887 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae0c229-9f8d-4017-b4b6-f57b050fc5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:12.205172 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0c229-9f8d-4017-b4b6-f57b050fc5a7 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:12.205657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0c229-9f8d-4017-b4b6-f57b050fc5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:12.205995 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0c229-9f8d-4017-b4b6-f57b050fc5a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:12.223127 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae0c229-9f8d-4017-b4b6-f57b050fc5a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:12.237828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-0ae0c229-9f8d-4017-b4b6-f57b050fc5a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:12.237828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:12.237828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:12.237828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:12.237828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:12.237828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:12.237828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:12.237828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:12.237828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:12.237828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:12.237828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:12.237828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:12.237828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:12.237828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:12.237828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:12.237828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:12.239802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:12.239802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:12.239802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:12.239802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:12.239802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:12.239802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:12.239802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:12.239802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:12.239802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:12.239802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:12.239802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:12.239802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:12.239802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:12.239802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:12.239802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:12.239802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:12.241849 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:12.241849 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:12.241849 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:12.241849 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:12.241849 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:12.241849 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:12.241849 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:12.241849 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:12.241849 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:12.241849 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:12.241849 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:12.241849 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:12.249957 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 417/848] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 15:59:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:12.249957 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae0c229-9f8d-4017-b4b6-f57b050fc5a7 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:12.311722 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 432/849] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:11 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 675 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 15:59:12.318088 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd7e40-2847-4994-98e8-df9589bd537b None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:12.448895 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae0c229-9f8d-4017-b4b6-f57b050fc5a7 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:12.515812 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd7e40-2847-4994-98e8-df9589bd537b None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:12.517076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ed935-c2e5-4ee6-9e66-41c96e5096b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:12.518271 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ed935-c2e5-4ee6-9e66-41c96e5096b6 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:12.518880 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ed935-c2e5-4ee6-9e66-41c96e5096b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:12.519177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ed935-c2e5-4ee6-9e66-41c96e5096b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:12.520316 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ed935-c2e5-4ee6-9e66-41c96e5096b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:12.520778 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ed935-c2e5-4ee6-9e66-41c96e5096b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:12.523875 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ed935-c2e5-4ee6-9e66-41c96e5096b6 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:12.542320 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 418/850] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 15:59:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:12.547893 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ed935-c2e5-4ee6-9e66-41c96e5096b6 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:12.552726 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:12.586880 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff29600-a6e9-435f-b26d-a0e550f15890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:12.587729 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff29600-a6e9-435f-b26d-a0e550f15890 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:12.588095 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff29600-a6e9-435f-b26d-a0e550f15890 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:12.588654 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff29600-a6e9-435f-b26d-a0e550f15890 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:12.610677 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff29600-a6e9-435f-b26d-a0e550f15890 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:12.611047 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:12.614250 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff29600-a6e9-435f-b26d-a0e550f15890 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:12.647333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-fff29600-a6e9-435f-b26d-a0e550f15890 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:12.647333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:12.647333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:12.647333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:12.647333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:12.647333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:12.647333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:12.647333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:12.647333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:12.647333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:12.647333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:12.647333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:12.647333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:12.647333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:12.647333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:12.648318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:12.648318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:12.648318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:12.648318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:12.648318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:12.648318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:12.648318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:12.648318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:12.648318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:12.648318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:12.648318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:12.648318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:12.648318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:12.648318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:12.648318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:12.648318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:12.648318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:12.649215 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:12.649215 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:12.649215 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:12.649215 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:12.650011 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 433/851] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:12.656396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff29600-a6e9-435f-b26d-a0e550f15890 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:12.687915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ed935-c2e5-4ee6-9e66-41c96e5096b6 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:12.734189 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3400578d-d014-4a14-b176-30634096e149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:12.734411 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400578d-d014-4a14-b176-30634096e149 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:12.734656 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400578d-d014-4a14-b176-30634096e149 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:12.734890 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400578d-d014-4a14-b176-30634096e149 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:12.747718 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3400578d-d014-4a14-b176-30634096e149 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:12.749790 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3400578d-d014-4a14-b176-30634096e149 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:12.771076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-3400578d-d014-4a14-b176-30634096e149 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:12.771076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:12.771076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:12.771076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:12.771076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:12.771076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:12.771076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:12.771076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:12.771076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:12.771076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:12.771076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:12.771076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:12.771076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:12.771076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:12.771076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:12.772046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:12.772046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:12.772046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:12.772046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:12.772046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:12.772046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:12.772046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:12.772046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:12.772046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:12.772046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:12.772046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:12.772046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:12.772046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:12.772046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:12.772046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:12.772046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:12.772046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:12.772941 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:12.772941 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:12.772941 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:12.772941 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:12.781206 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 419/852] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:12.781206 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3400578d-d014-4a14-b176-30634096e149 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:12.829890 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff29600-a6e9-435f-b26d-a0e550f15890 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:12.870291 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cca608-e674-4823-ac7b-9b8d7c2ec582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:12.870976 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cca608-e674-4823-ac7b-9b8d7c2ec582 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:12.871260 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cca608-e674-4823-ac7b-9b8d7c2ec582 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:12.871543 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cca608-e674-4823-ac7b-9b8d7c2ec582 None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:12.872284 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cca608-e674-4823-ac7b-9b8d7c2ec582 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:12.872593 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cca608-e674-4823-ac7b-9b8d7c2ec582 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:12.874074 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cca608-e674-4823-ac7b-9b8d7c2ec582 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:12.891803 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 434/853] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:12.898909 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cca608-e674-4823-ac7b-9b8d7c2ec582 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:12.932883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:12.954080 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:12.964526 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3400578d-d014-4a14-b176-30634096e149 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:13.020316 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb1188-498f-499c-93b3-677cc31f7270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:13.020925 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb1188-498f-499c-93b3-677cc31f7270 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:13.021211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb1188-498f-499c-93b3-677cc31f7270 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:13.021490 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb1188-498f-499c-93b3-677cc31f7270 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:13.022588 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb1188-498f-499c-93b3-677cc31f7270 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:13.022908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb1188-498f-499c-93b3-677cc31f7270 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:13.024505 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb1188-498f-499c-93b3-677cc31f7270 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:13.038091 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cca608-e674-4823-ac7b-9b8d7c2ec582 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:13.041814 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 420/854] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:13.050508 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb1188-498f-499c-93b3-677cc31f7270 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:13.087167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c8018-8e69-4557-9a3a-7c9dc5bcd048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:13.088375 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c8018-8e69-4557-9a3a-7c9dc5bcd048 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:13.088979 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c8018-8e69-4557-9a3a-7c9dc5bcd048 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:13.089474 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c8018-8e69-4557-9a3a-7c9dc5bcd048 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:13.100963 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c8018-8e69-4557-9a3a-7c9dc5bcd048 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:13.103615 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c8018-8e69-4557-9a3a-7c9dc5bcd048 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:13.124488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-ab5c8018-8e69-4557-9a3a-7c9dc5bcd048 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:13.124488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:13.124488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:13.124488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:13.124488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:13.124488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:13.124488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:13.124488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:13.124488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:13.124488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:13.124488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:13.124488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:13.124488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:13.124488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:13.124488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:13.125457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:13.125457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:13.125457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:13.125457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:13.125457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:13.125457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:13.125457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:13.125457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:13.125457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:13.125457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:13.125457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:13.125457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:13.125457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:13.125457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:13.125457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:13.125457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:13.125457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:13.126388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:13.126388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:13.126388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:13.126388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:13.126388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:13.126388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:13.126388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:13.126388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:13.134004 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 435/855] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:13.140035 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c8018-8e69-4557-9a3a-7c9dc5bcd048 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:13.221138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb1188-498f-499c-93b3-677cc31f7270 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:13.272252 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510dd883-6682-47d3-87ad-337a6411c0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:13.273571 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dd883-6682-47d3-87ad-337a6411c0f5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:13.273958 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dd883-6682-47d3-87ad-337a6411c0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:13.274396 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dd883-6682-47d3-87ad-337a6411c0f5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:13.282744 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:13.286504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510dd883-6682-47d3-87ad-337a6411c0f5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:13.288264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510dd883-6682-47d3-87ad-337a6411c0f5 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:13.308986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-510dd883-6682-47d3-87ad-337a6411c0f5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 15:59:13.308986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:13.308986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:13.308986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:13.308986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:13.308986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:13.308986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:13.308986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:13.308986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:13.308986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:13.308986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:13.308986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:13.308986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:13.308986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:13.308986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:13.308986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:13.312510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:13.312510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:13.312510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:13.312510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:13.312510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:13.312510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:13.312510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:13.312510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:13.312510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:13.312510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:13.312510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:13.312510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:13.312510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:13.312510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:13.312510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:13.312510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:13.312510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:13.312510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:13.314481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:13.314481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:13.314481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:13.314481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:13.314481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:13.314481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:13.314481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:13.314481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:13.314481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 15:59:13.314481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:13.317702 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c8018-8e69-4557-9a3a-7c9dc5bcd048 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:13.318604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 421/856] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jun 16 15:59:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:13.320753 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510dd883-6682-47d3-87ad-337a6411c0f5 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:13.360459 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:13.364852 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb1199-6fe1-4f50-877b-2102a0d7601f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:13.365544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb1199-6fe1-4f50-877b-2102a0d7601f None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:13.365822 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb1199-6fe1-4f50-877b-2102a0d7601f None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:13.366153 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb1199-6fe1-4f50-877b-2102a0d7601f None admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:13.367168 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb1199-6fe1-4f50-877b-2102a0d7601f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:13.367489 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb1199-6fe1-4f50-877b-2102a0d7601f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:13.368976 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb1199-6fe1-4f50-877b-2102a0d7601f None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:13.388766 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 436/857] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:13.390005 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb1199-6fe1-4f50-877b-2102a0d7601f None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:13.470548 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510dd883-6682-47d3-87ad-337a6411c0f5 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:13.506149 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb1199-6fe1-4f50-877b-2102a0d7601f None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:13.531380 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d1b5d-5b0d-4a04-9674-5d2cd5a65fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:13.533175 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d1b5d-5b0d-4a04-9674-5d2cd5a65fef None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:13.534101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d1b5d-5b0d-4a04-9674-5d2cd5a65fef None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:13.535112 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d1b5d-5b0d-4a04-9674-5d2cd5a65fef None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:13.537362 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d1b5d-5b0d-4a04-9674-5d2cd5a65fef None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:13.538435 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d1b5d-5b0d-4a04-9674-5d2cd5a65fef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:13.542138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d1b5d-5b0d-4a04-9674-5d2cd5a65fef None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:13.550875 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a186a63-0049-410a-a1e9-124fa143e9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:13.551174 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a186a63-0049-410a-a1e9-124fa143e9c7 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:13.551313 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a186a63-0049-410a-a1e9-124fa143e9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:13.551719 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a186a63-0049-410a-a1e9-124fa143e9c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:13.552732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a186a63-0049-410a-a1e9-124fa143e9c7 None admin] RBAC: Adding query filter params (role.id=bbbc77ec645d4ff5a8bf29ce0377f578, scope.project.id=b1177a85a0de4276898daf4a24ff069f, user.id=0082e37ce54f49f7837601349508c89f) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:13.553072 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a186a63-0049-410a-a1e9-124fa143e9c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:13.554779 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a186a63-0049-410a-a1e9-124fa143e9c7 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:13.570410 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 437/858] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:59:13 2022] GET /identity/v3/role_assignments?user.id=0082e37ce54f49f7837601349508c89f&scope.project.id=b1177a85a0de4276898daf4a24ff069f&role.id=bbbc77ec645d4ff5a8bf29ce0377f578 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:13.587381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 422/859] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jun 16 15:59:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:13.593700 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a186a63-0049-410a-a1e9-124fa143e9c7 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:13.606817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e7380-e79f-4365-84c5-bc744096c226 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:13.607197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e7380-e79f-4365-84c5-bc744096c226 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:13.607386 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e7380-e79f-4365-84c5-bc744096c226 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:13.610787 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 423/860] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:59:13.614479 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:13.615240 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fcdaf-43f1-4750-b4ed-958611b3f36e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:13.615528 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fcdaf-43f1-4750-b4ed-958611b3f36e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:13.615810 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fcdaf-43f1-4750-b4ed-958611b3f36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:13.639173 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-2d1fcdaf-43f1-4750-b4ed-958611b3f36e None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:13.640604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2d1fcdaf-43f1-4750-b4ed-958611b3f36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:13.721762 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a186a63-0049-410a-a1e9-124fa143e9c7 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:13.746613 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 424/861] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:13.763025 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fcdaf-43f1-4750-b4ed-958611b3f36e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:13.766186 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:13.781916 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97efa839-249f-4fcc-a8c5-8e23983bcf00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:13.783580 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efa839-249f-4fcc-a8c5-8e23983bcf00 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:13.784020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efa839-249f-4fcc-a8c5-8e23983bcf00 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:13.784352 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efa839-249f-4fcc-a8c5-8e23983bcf00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:13.892747 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97efa839-249f-4fcc-a8c5-8e23983bcf00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:13.894165 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97efa839-249f-4fcc-a8c5-8e23983bcf00 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:13.911779 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fcdaf-43f1-4750-b4ed-958611b3f36e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:13.914658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:13.959617 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76730d-ef55-4c3e-a33b-8897cebea15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:13.960397 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76730d-ef55-4c3e-a33b-8897cebea15a None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:13.960748 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76730d-ef55-4c3e-a33b-8897cebea15a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:13.961089 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76730d-ef55-4c3e-a33b-8897cebea15a None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:13.961890 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76730d-ef55-4c3e-a33b-8897cebea15a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:13.963342 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76730d-ef55-4c3e-a33b-8897cebea15a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:13.979197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 425/862] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jun 16 15:59:13 2022] POST /identity/v3/services => generated 251 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:59:13.980538 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5360f8-5982-4c05-b8b2-7bcf17fd209e None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:13.980827 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5360f8-5982-4c05-b8b2-7bcf17fd209e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:13.981102 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5360f8-5982-4c05-b8b2-7bcf17fd209e None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:13.983335 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 426/863] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:13.993465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c70624-9a1a-44e5-9f4d-0b2b044ffd77 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:13.994985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c70624-9a1a-44e5-9f4d-0b2b044ffd77 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:13.995269 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c70624-9a1a-44e5-9f4d-0b2b044ffd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:14.018056 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d0c70624-9a1a-44e5-9f4d-0b2b044ffd77 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:14.019609 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d0c70624-9a1a-44e5-9f4d-0b2b044ffd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:14.040773 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 438/864] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:13 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 15:59:14.047549 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97efa839-249f-4fcc-a8c5-8e23983bcf00 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:14.106685 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:14.140771 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 427/865] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:13 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:14.164485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c70624-9a1a-44e5-9f4d-0b2b044ffd77 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:14.200454 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97efa839-249f-4fcc-a8c5-8e23983bcf00 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:14.221095 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:14.251432 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5760a1-a304-4b2a-bad9-a2aab4eaea09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:14.251432 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5760a1-a304-4b2a-bad9-a2aab4eaea09 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:14.251432 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5760a1-a304-4b2a-bad9-a2aab4eaea09 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:14.251779 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5760a1-a304-4b2a-bad9-a2aab4eaea09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:14.270607 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5760a1-a304-4b2a-bad9-a2aab4eaea09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:14.272299 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5760a1-a304-4b2a-bad9-a2aab4eaea09 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:14.293208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-8a5760a1-a304-4b2a-bad9-a2aab4eaea09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:14.293208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:14.293208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:14.293208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:14.293208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:14.293208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:14.293208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:14.293208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:14.293208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:14.293208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:14.293208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:14.293208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:14.293208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:14.293208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:14.293208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:14.294148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:14.294148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:14.294148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:14.294148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:14.294148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:14.294148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:14.294148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:14.294148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:14.294148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:14.294148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:14.294148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:14.294148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:14.294148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:14.294148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:14.294148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:14.294148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:14.294148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:14.295600 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:14.295600 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:14.295600 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:14.295600 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:14.295600 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 439/866] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:14.302433 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5760a1-a304-4b2a-bad9-a2aab4eaea09 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:14.328816 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c70624-9a1a-44e5-9f4d-0b2b044ffd77 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:14.391908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154f914-8664-4e17-be99-5c5d0b2d3d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:14.392788 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154f914-8664-4e17-be99-5c5d0b2d3d27 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:14.393069 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154f914-8664-4e17-be99-5c5d0b2d3d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:14.393343 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154f914-8664-4e17-be99-5c5d0b2d3d27 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:14.394042 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a154f914-8664-4e17-be99-5c5d0b2d3d27 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:14.395461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a154f914-8664-4e17-be99-5c5d0b2d3d27 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:14.410282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-a154f914-8664-4e17-be99-5c5d0b2d3d27 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 15:59:14.410282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:14.410282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 15:59:14.410282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 15:59:14.410282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 15:59:14.410282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 15:59:14.410282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 15:59:14.410282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:14.410282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 15:59:14.410282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:14.410282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:59:14.410282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:14.410282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:14.410282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:14.410282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:14.410282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:14.411368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:14.411368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:14.411368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:14.411368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:14.411368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:14.411368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:14.411368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:14.411368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 15:59:14.411368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 15:59:14.411368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 15:59:14.411368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 15:59:14.411368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:14.411368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:14.411368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:14.411368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:14.411368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:14.412249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:14.412249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:14.412249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:14.412249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:14.412249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:14.412249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:14.412249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:14.412249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:14.412249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:14.412249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:14.412249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:14.412249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 15:59:14.412249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:14.412249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 15:59:14.412249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:14.412249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 428/867] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jun 16 15:59:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:14.419220 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154f914-8664-4e17-be99-5c5d0b2d3d27 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:14.474210 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5760a1-a304-4b2a-bad9-a2aab4eaea09 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:14.496474 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:14.537836 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082bd2db-4a84-4dc8-a2ef-cecb22944623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:14.542782 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082bd2db-4a84-4dc8-a2ef-cecb22944623 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:14.542782 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082bd2db-4a84-4dc8-a2ef-cecb22944623 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:14.543104 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082bd2db-4a84-4dc8-a2ef-cecb22944623 None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:14.543636 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082bd2db-4a84-4dc8-a2ef-cecb22944623 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:14.543883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082bd2db-4a84-4dc8-a2ef-cecb22944623 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:14.546224 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082bd2db-4a84-4dc8-a2ef-cecb22944623 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:14.569479 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 440/868] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:14.571485 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082bd2db-4a84-4dc8-a2ef-cecb22944623 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:14.603086 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154f914-8664-4e17-be99-5c5d0b2d3d27 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:14.621166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:14.681192 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757238ef-0d49-4026-8fd3-0388918a4279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:14.682472 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757238ef-0d49-4026-8fd3-0388918a4279 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:14.682916 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757238ef-0d49-4026-8fd3-0388918a4279 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:14.683353 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757238ef-0d49-4026-8fd3-0388918a4279 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:14.684259 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757238ef-0d49-4026-8fd3-0388918a4279 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:14.684709 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757238ef-0d49-4026-8fd3-0388918a4279 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:14.686639 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757238ef-0d49-4026-8fd3-0388918a4279 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:14.692451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 429/869] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jun 16 15:59:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:14.703429 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757238ef-0d49-4026-8fd3-0388918a4279 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:14.743537 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082bd2db-4a84-4dc8-a2ef-cecb22944623 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:14.810357 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfab75f-3795-48b0-a862-1345961a6b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:14.812886 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfab75f-3795-48b0-a862-1345961a6b8d None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:14.813662 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfab75f-3795-48b0-a862-1345961a6b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:14.814450 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfab75f-3795-48b0-a862-1345961a6b8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:14.828198 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfab75f-3795-48b0-a862-1345961a6b8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:14.831337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfab75f-3795-48b0-a862-1345961a6b8d None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:14.850350 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-8dfab75f-3795-48b0-a862-1345961a6b8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:14.850350 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:14.850350 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:14.850350 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:14.850350 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:14.850350 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:14.850350 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:14.850350 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:14.850350 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:14.850350 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:14.850350 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:14.850350 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:14.850350 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:14.850350 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:14.850350 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:14.851336 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:14.851336 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:14.851336 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:14.851336 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:14.851336 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:14.851336 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:14.851336 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:14.851336 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:14.851336 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:14.851336 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:14.851336 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:14.851336 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:14.851336 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:14.851336 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:14.851336 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:14.851336 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:14.851336 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:14.852213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:14.852213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:14.852213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:14.852213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:14.852213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:14.852213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:14.852213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:14.852213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:14.854922 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 441/870] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:14.868234 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfab75f-3795-48b0-a862-1345961a6b8d None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:14.903629 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757238ef-0d49-4026-8fd3-0388918a4279 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:14.926569 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:14.948736 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4d98c-ccec-42b3-bc84-6bfe6e2f2f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:14.949517 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4d98c-ccec-42b3-bc84-6bfe6e2f2f54 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:14.949808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4d98c-ccec-42b3-bc84-6bfe6e2f2f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:14.950126 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4d98c-ccec-42b3-bc84-6bfe6e2f2f54 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:14.950883 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4d98c-ccec-42b3-bc84-6bfe6e2f2f54 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:14.951195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4d98c-ccec-42b3-bc84-6bfe6e2f2f54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:14.953248 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4d98c-ccec-42b3-bc84-6bfe6e2f2f54 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:14.960336 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 430/871] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jun 16 15:59:14 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:15.005578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:15.027560 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfab75f-3795-48b0-a862-1345961a6b8d None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:15.079371 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43fbc8c-7430-4326-a5f5-bef2a631995d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:15.079986 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43fbc8c-7430-4326-a5f5-bef2a631995d None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:15.080276 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43fbc8c-7430-4326-a5f5-bef2a631995d None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:15.080557 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43fbc8c-7430-4326-a5f5-bef2a631995d None admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:15.081393 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43fbc8c-7430-4326-a5f5-bef2a631995d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:15.081713 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43fbc8c-7430-4326-a5f5-bef2a631995d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:15.083403 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43fbc8c-7430-4326-a5f5-bef2a631995d None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:15.100805 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 442/872] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:15.108557 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4d98c-ccec-42b3-bc84-6bfe6e2f2f54 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:15.269517 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4d98c-ccec-42b3-bc84-6bfe6e2f2f54 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:15.329602 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7be7ec-350c-4692-9823-008d5e40ff9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:15.330255 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7be7ec-350c-4692-9823-008d5e40ff9c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:15.330547 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7be7ec-350c-4692-9823-008d5e40ff9c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:15.330829 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7be7ec-350c-4692-9823-008d5e40ff9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:15.331698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7be7ec-350c-4692-9823-008d5e40ff9c None admin] RBAC: Adding query filter params (role.id=cfc67444e9944ffc87173be8c9245108, scope.project.id=b1177a85a0de4276898daf4a24ff069f, user.id=2c2e0e19880f4b048faf034dd38c7365) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:15.332010 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7be7ec-350c-4692-9823-008d5e40ff9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:15.333507 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7be7ec-350c-4692-9823-008d5e40ff9c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:15.341472 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 431/873] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:59:15 2022] GET /identity/v3/role_assignments?user.id=2c2e0e19880f4b048faf034dd38c7365&scope.project.id=b1177a85a0de4276898daf4a24ff069f&role.id=cfc67444e9944ffc87173be8c9245108 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:16.554725 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040a6f4-fe51-431b-a243-9156d483f48c None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:16.555799 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040a6f4-fe51-431b-a243-9156d483f48c None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:16.556298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040a6f4-fe51-431b-a243-9156d483f48c None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:16.558427 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 443/874] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:16.565315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db66d8e-a4ed-45bb-a741-b73eaa4fd286 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:16.566003 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db66d8e-a4ed-45bb-a741-b73eaa4fd286 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:16.566003 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db66d8e-a4ed-45bb-a741-b73eaa4fd286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:16.586847 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-3db66d8e-a4ed-45bb-a741-b73eaa4fd286 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:16.589093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3db66d8e-a4ed-45bb-a741-b73eaa4fd286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:16.648229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:16.694110 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 432/875] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:16 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:16.719904 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7040a6f4-fe51-431b-a243-9156d483f48c None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:16.735801 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2df758-0d92-480e-849b-82521b2b2c6a None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:16.736289 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2df758-0d92-480e-849b-82521b2b2c6a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:16.736610 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2df758-0d92-480e-849b-82521b2b2c6a None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:16.738351 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 433/876] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:59:16.743992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895bdfc-de5c-4ea1-8057-37e0180feb7f None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:16.744584 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895bdfc-de5c-4ea1-8057-37e0180feb7f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:16.745081 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895bdfc-de5c-4ea1-8057-37e0180feb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:16.764933 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8895bdfc-de5c-4ea1-8057-37e0180feb7f None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:16.766779 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8895bdfc-de5c-4ea1-8057-37e0180feb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:16.869992 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7040a6f4-fe51-431b-a243-9156d483f48c None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:16.916447 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d6f04-5667-463c-8261-7563a2556770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:16.917637 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d6f04-5667-463c-8261-7563a2556770 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:16.918166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d6f04-5667-463c-8261-7563a2556770 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:16.918670 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d6f04-5667-463c-8261-7563a2556770 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:16.919575 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d6f04-5667-463c-8261-7563a2556770 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:16.922182 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d6f04-5667-463c-8261-7563a2556770 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:16.931782 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 434/877] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:16 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 15:59:16.942512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-9d2d6f04-5667-463c-8261-7563a2556770 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 15:59:16.942512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:16.942512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 15:59:16.942512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 15:59:16.942512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 15:59:16.942512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 15:59:16.942512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 15:59:16.942512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:16.942512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 15:59:16.942512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:16.942512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:59:16.942512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:16.942512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:16.942512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:16.942512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:16.942512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:16.942512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:16.944390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:16.944390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:16.944390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:16.944390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:16.944390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:16.944390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:16.944390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 15:59:16.944390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 15:59:16.944390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 15:59:16.944390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 15:59:16.944390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:16.944390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:16.944390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:16.944390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:16.944390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:16.944390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:16.946006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:16.946006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:16.946006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:16.946006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:16.946006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:16.946006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:16.946006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:16.946006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:16.946006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:16.946006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:16.946006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 15:59:16.946006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:16.946006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 15:59:16.946006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:16.954080 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 444/878] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jun 16 15:59:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:16.956180 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895bdfc-de5c-4ea1-8057-37e0180feb7f None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:16.961857 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d6f04-5667-463c-8261-7563a2556770 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:16.980185 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:17.095307 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d6f04-5667-463c-8261-7563a2556770 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:17.137410 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db2281-89df-4485-96b8-c9e07d80ef28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:17.138063 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db2281-89df-4485-96b8-c9e07d80ef28 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:17.138345 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db2281-89df-4485-96b8-c9e07d80ef28 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:17.138647 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db2281-89df-4485-96b8-c9e07d80ef28 None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:17.139638 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db2281-89df-4485-96b8-c9e07d80ef28 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:17.140186 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db2281-89df-4485-96b8-c9e07d80ef28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:17.141684 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db2281-89df-4485-96b8-c9e07d80ef28 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:17.149745 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 445/879] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jun 16 15:59:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:17.159286 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db2281-89df-4485-96b8-c9e07d80ef28 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:17.183960 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895bdfc-de5c-4ea1-8057-37e0180feb7f None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:17.190207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:17.247802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:17.260596 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49f2b9-d1c9-47a2-afbc-ccd8d88450f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:17.262343 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49f2b9-d1c9-47a2-afbc-ccd8d88450f6 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:17.263066 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49f2b9-d1c9-47a2-afbc-ccd8d88450f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:17.263755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49f2b9-d1c9-47a2-afbc-ccd8d88450f6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:17.266033 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49f2b9-d1c9-47a2-afbc-ccd8d88450f6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:17.268902 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49f2b9-d1c9-47a2-afbc-ccd8d88450f6 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:17.287533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db2281-89df-4485-96b8-c9e07d80ef28 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:17.293442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-7a49f2b9-d1c9-47a2-afbc-ccd8d88450f6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 15:59:17.293442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:17.293442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 15:59:17.293442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 15:59:17.293442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 15:59:17.293442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 15:59:17.293442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 15:59:17.293442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:17.293442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 15:59:17.293442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:17.293442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:59:17.293442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:17.293442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:17.293442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:17.293442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:17.293442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:17.293442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:17.296224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:17.296224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:17.296224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:17.296224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:17.296224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:17.296224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:17.296224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 15:59:17.296224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 15:59:17.296224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 15:59:17.296224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 15:59:17.296224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:17.296224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:17.296224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:17.296224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:17.296224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:17.296224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:17.298235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:17.298235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:17.298235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:17.298235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:17.298235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:17.298235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:17.298235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:17.298235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:17.298235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:17.298235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:17.298235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 15:59:17.298235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:17.298235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 15:59:17.298235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:17.303379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 435/880] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 15:59:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:17.311566 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49f2b9-d1c9-47a2-afbc-ccd8d88450f6 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:17.340832 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65b51c6-48b1-4985-ae47-751ee67e2a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:17.342986 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b51c6-48b1-4985-ae47-751ee67e2a8d None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:17.343737 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b51c6-48b1-4985-ae47-751ee67e2a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:17.344235 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b51c6-48b1-4985-ae47-751ee67e2a8d None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:17.345271 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65b51c6-48b1-4985-ae47-751ee67e2a8d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:17.345811 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65b51c6-48b1-4985-ae47-751ee67e2a8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:17.347503 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65b51c6-48b1-4985-ae47-751ee67e2a8d None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:17.359386 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 446/881] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jun 16 15:59:17 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:17.551962 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49f2b9-d1c9-47a2-afbc-ccd8d88450f6 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:17.600113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c61363-515d-4b61-a017-1d7103d8b1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:17.601593 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c61363-515d-4b61-a017-1d7103d8b1ee None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:17.602128 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c61363-515d-4b61-a017-1d7103d8b1ee None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:17.602605 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c61363-515d-4b61-a017-1d7103d8b1ee None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:17.603525 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c61363-515d-4b61-a017-1d7103d8b1ee None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:17.604081 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c61363-515d-4b61-a017-1d7103d8b1ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:17.605712 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c61363-515d-4b61-a017-1d7103d8b1ee None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:17.617090 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 436/882] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 15:59:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:17.618636 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65b51c6-48b1-4985-ae47-751ee67e2a8d None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:17.765490 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743cfc6d-6ab7-42ef-9ffa-97c19eb80729 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:17.765911 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743cfc6d-6ab7-42ef-9ffa-97c19eb80729 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:17.766115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743cfc6d-6ab7-42ef-9ffa-97c19eb80729 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:17.767897 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 437/883] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:17.773393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b102be8-516e-42ad-a281-6ddb3f804abf None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:17.773694 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b102be8-516e-42ad-a281-6ddb3f804abf None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:17.774269 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b102be8-516e-42ad-a281-6ddb3f804abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:17.783911 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65b51c6-48b1-4985-ae47-751ee67e2a8d None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:17.798107 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4b102be8-516e-42ad-a281-6ddb3f804abf None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:17.800131 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4b102be8-516e-42ad-a281-6ddb3f804abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:17.836856 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:17.846370 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6cacdd-d49a-4ff2-8a5c-26d4844c5d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:17.847547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6cacdd-d49a-4ff2-8a5c-26d4844c5d87 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:17.848046 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6cacdd-d49a-4ff2-8a5c-26d4844c5d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:17.848520 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6cacdd-d49a-4ff2-8a5c-26d4844c5d87 None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:17.850115 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6cacdd-d49a-4ff2-8a5c-26d4844c5d87 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:17.850663 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6cacdd-d49a-4ff2-8a5c-26d4844c5d87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:17.853577 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6cacdd-d49a-4ff2-8a5c-26d4844c5d87 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:17.857990 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:17.870136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 447/884] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 15:59:17 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:17.872004 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255636bb-049f-4731-888f-8b35a2c11b4e None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:17.872564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255636bb-049f-4731-888f-8b35a2c11b4e None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:17.872863 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255636bb-049f-4731-888f-8b35a2c11b4e None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:17.874603 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 448/885] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:17.875245 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255636bb-049f-4731-888f-8b35a2c11b4e None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:17.923633 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 438/886] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:17 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:17.926107 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d98a6c-a02b-4894-9e96-d087771522d3 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:17.926597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d98a6c-a02b-4894-9e96-d087771522d3 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:17.927060 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d98a6c-a02b-4894-9e96-d087771522d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:17.962323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-33d98a6c-a02b-4894-9e96-d087771522d3 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:17.964065 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-33d98a6c-a02b-4894-9e96-d087771522d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:18.062091 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255636bb-049f-4731-888f-8b35a2c11b4e None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:18.093295 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 439/887] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:17 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:18.093295 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d98a6c-a02b-4894-9e96-d087771522d3 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:18.126241 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3ffdd-e226-481c-9ee6-8044c42a2aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:18.127346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3ffdd-e226-481c-9ee6-8044c42a2aee None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:18.127868 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3ffdd-e226-481c-9ee6-8044c42a2aee None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:18.128324 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3ffdd-e226-481c-9ee6-8044c42a2aee None admin] PATH_INFO: `/v3/endpoints` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:18.129832 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd3ffdd-e226-481c-9ee6-8044c42a2aee None admin] RBAC: Adding query filter params (interface=public, service_id=e688dac2bb63418688110e3c81c43dbf, region_id=RegionOne) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:18.130434 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd3ffdd-e226-481c-9ee6-8044c42a2aee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:18.133140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd3ffdd-e226-481c-9ee6-8044c42a2aee None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:18.145604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 449/888] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jun 16 15:59:17 2022] GET /identity/v3/endpoints?service_id=e688dac2bb63418688110e3c81c43dbf&interface=public®ion_id=RegionOne => generated 194 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:18.145604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3ffdd-e226-481c-9ee6-8044c42a2aee None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:18.246887 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:18.254948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:18.273662 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d98a6c-a02b-4894-9e96-d087771522d3 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:18.300254 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3ffdd-e226-481c-9ee6-8044c42a2aee None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:18.330566 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ab278-fbc3-4c74-98ad-be073c7b397f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:18.331698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ab278-fbc3-4c74-98ad-be073c7b397f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:18.332104 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ab278-fbc3-4c74-98ad-be073c7b397f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:18.332495 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ab278-fbc3-4c74-98ad-be073c7b397f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:18.347300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ab278-fbc3-4c74-98ad-be073c7b397f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:18.355949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-f68ab278-fbc3-4c74-98ad-be073c7b397f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 15:59:18.355949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:18.355949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:18.355949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:18.355949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:18.355949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:18.355949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:18.355949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:18.355949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:18.355949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:18.355949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:18.355949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:18.355949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:18.355949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:18.355949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:18.355949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:18.357393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:18.357393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:18.357393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:18.357393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:18.357393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:18.357393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:18.357393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:18.357393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:18.357393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:18.357393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:18.357393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:18.357393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:18.357393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:18.357393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:18.357393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:18.357393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:18.358663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:18.358663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:18.358663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:18.358663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:18.358663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:18.358663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:18.358663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:18.358663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:18.358663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:18.358663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:18.358663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 15:59:18.358663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:18.358663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 440/889] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jun 16 15:59:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:18.358663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ab278-fbc3-4c74-98ad-be073c7b397f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:18.360173 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342074ab-434d-42e5-bb90-b90c8baf360c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:18.360856 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342074ab-434d-42e5-bb90-b90c8baf360c None admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:18.361132 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342074ab-434d-42e5-bb90-b90c8baf360c None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:18.361435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342074ab-434d-42e5-bb90-b90c8baf360c None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:18.362205 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342074ab-434d-42e5-bb90-b90c8baf360c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:18.363696 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342074ab-434d-42e5-bb90-b90c8baf360c None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:18.383486 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 450/890] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jun 16 15:59:18 2022] POST /identity/v3/services => generated 269 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:59:18.385861 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342074ab-434d-42e5-bb90-b90c8baf360c None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:18.508510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ab278-fbc3-4c74-98ad-be073c7b397f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:18.583092 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2670f029-cf1a-4d61-86e3-210c5080306a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:18.583786 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670f029-cf1a-4d61-86e3-210c5080306a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:18.584080 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670f029-cf1a-4d61-86e3-210c5080306a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:18.584332 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670f029-cf1a-4d61-86e3-210c5080306a None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:18.585020 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2670f029-cf1a-4d61-86e3-210c5080306a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:18.585699 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342074ab-434d-42e5-bb90-b90c8baf360c None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:18.586482 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2670f029-cf1a-4d61-86e3-210c5080306a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:18.598231 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 441/891] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jun 16 15:59:18 2022] GET /identity/v3/services => generated 1268 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:59:18.625061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:18.651554 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3009f6a-5d25-4213-95bc-54b31f91c915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:18.653341 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3009f6a-5d25-4213-95bc-54b31f91c915 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:18.654304 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3009f6a-5d25-4213-95bc-54b31f91c915 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:18.655692 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3009f6a-5d25-4213-95bc-54b31f91c915 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:18.657710 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3009f6a-5d25-4213-95bc-54b31f91c915 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:18.658752 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3009f6a-5d25-4213-95bc-54b31f91c915 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:18.661878 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3009f6a-5d25-4213-95bc-54b31f91c915 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:18.680362 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 451/892] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jun 16 15:59:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:18.686684 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2670f029-cf1a-4d61-86e3-210c5080306a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:18.708466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:18.847014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2670f029-cf1a-4d61-86e3-210c5080306a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:18.913383 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b4019-f53f-47e9-adf1-1da784505ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:18.915295 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b4019-f53f-47e9-adf1-1da784505ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:18.916161 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b4019-f53f-47e9-adf1-1da784505ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:18.916944 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b4019-f53f-47e9-adf1-1da784505ce2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:18.939760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51b4019-f53f-47e9-adf1-1da784505ce2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:18.943036 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51b4019-f53f-47e9-adf1-1da784505ce2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:18.978235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-f51b4019-f53f-47e9-adf1-1da784505ce2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:18.978235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:18.978235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:18.978235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:18.978235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:18.978235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:18.978235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:18.978235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:18.978235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:18.978235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:18.978235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:18.978235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:18.978235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:18.978235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:18.978235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:18.979980 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:18.979980 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:18.979980 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:18.979980 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:18.979980 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:18.979980 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:18.979980 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:18.979980 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:18.979980 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:18.979980 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:18.979980 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:18.979980 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:18.979980 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:18.979980 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:18.979980 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:18.979980 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:18.979980 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:18.981779 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:18.981779 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:18.981779 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:18.981779 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:18.985188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 442/893] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:18.991432 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3009f6a-5d25-4213-95bc-54b31f91c915 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:19.134622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3009f6a-5d25-4213-95bc-54b31f91c915 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:19.178458 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd017a3-ed67-430e-894f-194ffdb7c4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:19.179066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd017a3-ed67-430e-894f-194ffdb7c4fc None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:19.179352 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd017a3-ed67-430e-894f-194ffdb7c4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:19.179630 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd017a3-ed67-430e-894f-194ffdb7c4fc None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:19.180388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd017a3-ed67-430e-894f-194ffdb7c4fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:19.180698 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd017a3-ed67-430e-894f-194ffdb7c4fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:19.182165 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd017a3-ed67-430e-894f-194ffdb7c4fc None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:19.197496 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 452/894] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:19.205070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b4019-f53f-47e9-adf1-1da784505ce2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:19.290493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:19.337448 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b4019-f53f-47e9-adf1-1da784505ce2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:19.379028 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803a9ea-f223-4b26-8476-3d76ddf13671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:19.380275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803a9ea-f223-4b26-8476-3d76ddf13671 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:19.380780 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803a9ea-f223-4b26-8476-3d76ddf13671 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:19.381267 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803a9ea-f223-4b26-8476-3d76ddf13671 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:19.391661 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c803a9ea-f223-4b26-8476-3d76ddf13671 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:19.393545 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c803a9ea-f223-4b26-8476-3d76ddf13671 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:19.410585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-c803a9ea-f223-4b26-8476-3d76ddf13671 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 15:59:19.410585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:19.410585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:19.410585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:19.410585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:19.410585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:19.410585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:19.410585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:19.410585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:19.410585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:19.410585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:19.410585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:19.410585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:19.410585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:19.410585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:19.410585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:19.411619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:19.411619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:19.411619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:19.411619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:19.411619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:19.411619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:19.411619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:19.411619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:19.411619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:19.411619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:19.411619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:19.411619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:19.411619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:19.411619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:19.411619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:19.411619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:19.411619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:19.411619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:19.412573 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:19.412573 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:19.412573 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:19.412573 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:19.412573 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:19.412573 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:19.412573 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:19.412573 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:19.412573 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 15:59:19.412573 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:19.415396 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 443/895] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jun 16 15:59:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:19.423220 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd017a3-ed67-430e-894f-194ffdb7c4fc None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:19.492209 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:19.554972 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd017a3-ed67-430e-894f-194ffdb7c4fc None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:19.592572 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2e3a1-6601-4dd8-8d53-2ae62ef3f3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:19.593296 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2e3a1-6601-4dd8-8d53-2ae62ef3f3df None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:19.593586 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2e3a1-6601-4dd8-8d53-2ae62ef3f3df None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:19.593857 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2e3a1-6601-4dd8-8d53-2ae62ef3f3df None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:19.594739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a2e3a1-6601-4dd8-8d53-2ae62ef3f3df None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:19.595051 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a2e3a1-6601-4dd8-8d53-2ae62ef3f3df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:19.596501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a2e3a1-6601-4dd8-8d53-2ae62ef3f3df None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:19.625727 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 453/896] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jun 16 15:59:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:19.631844 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803a9ea-f223-4b26-8476-3d76ddf13671 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:19.748961 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803a9ea-f223-4b26-8476-3d76ddf13671 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:19.774708 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:19.788064 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b99a6-fffc-48ca-b9f2-df25f435eb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:19.789604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b99a6-fffc-48ca-b9f2-df25f435eb64 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:19.790429 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b99a6-fffc-48ca-b9f2-df25f435eb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:19.791187 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b99a6-fffc-48ca-b9f2-df25f435eb64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:19.870628 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b99a6-fffc-48ca-b9f2-df25f435eb64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:19.872174 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b99a6-fffc-48ca-b9f2-df25f435eb64 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:19.946197 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c92c77-86d7-4884-a0ae-909419e99fa4 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:19.946531 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c92c77-86d7-4884-a0ae-909419e99fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:19.946766 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c92c77-86d7-4884-a0ae-909419e99fa4 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:19.949483 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 454/897] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:19.954046 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e3ff8-f3c1-499e-b4a3-0abf71fc9502 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:19.954346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e3ff8-f3c1-499e-b4a3-0abf71fc9502 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:19.954625 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e3ff8-f3c1-499e-b4a3-0abf71fc9502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:19.972106 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-563e3ff8-f3c1-499e-b4a3-0abf71fc9502 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:19.973316 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-563e3ff8-f3c1-499e-b4a3-0abf71fc9502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:19.981334 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 444/898] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:19 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 15:59:19.989277 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b99a6-fffc-48ca-b9f2-df25f435eb64 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:20.018144 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:20.058891 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 455/899] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:19 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 15:59:20.082846 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e3ff8-f3c1-499e-b4a3-0abf71fc9502 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:20.120001 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:20.125196 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b99a6-fffc-48ca-b9f2-df25f435eb64 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:20.166732 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0493ca-b894-4250-ac0a-6159ce625092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:20.167364 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0493ca-b894-4250-ac0a-6159ce625092 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:20.167646 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0493ca-b894-4250-ac0a-6159ce625092 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:20.167924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0493ca-b894-4250-ac0a-6159ce625092 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:20.179642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0493ca-b894-4250-ac0a-6159ce625092 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:20.181100 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0493ca-b894-4250-ac0a-6159ce625092 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:20.200465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-aa0493ca-b894-4250-ac0a-6159ce625092 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:20.200465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:20.200465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:20.200465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:20.200465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:20.200465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:20.200465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:20.200465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:20.200465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:20.200465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:20.200465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:20.200465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:20.200465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:20.200465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:20.200465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:20.202232 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:20.202232 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:20.202232 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:20.202232 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:20.202232 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:20.202232 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:20.202232 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:20.202232 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:20.202232 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:20.202232 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:20.202232 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:20.202232 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:20.202232 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:20.202232 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:20.202232 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:20.202232 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:20.202232 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:20.203155 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:20.203155 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:20.203155 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:20.203155 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:20.203155 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 445/900] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:20.208404 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0493ca-b894-4250-ac0a-6159ce625092 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:20.210907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e3ff8-f3c1-499e-b4a3-0abf71fc9502 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:20.252634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c294104-ce4e-44c5-b6f2-dfa503f8e4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:20.253610 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c294104-ce4e-44c5-b6f2-dfa503f8e4a6 None admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:20.254245 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c294104-ce4e-44c5-b6f2-dfa503f8e4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:20.254763 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c294104-ce4e-44c5-b6f2-dfa503f8e4a6 None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:20.255739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c294104-ce4e-44c5-b6f2-dfa503f8e4a6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:20.257481 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c294104-ce4e-44c5-b6f2-dfa503f8e4a6 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:20.275944 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 456/901] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jun 16 15:59:20 2022] POST /identity/v3/services => generated 241 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:59:20.310331 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:20.341845 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0493ca-b894-4250-ac0a-6159ce625092 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:20.391997 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc370560-b003-4982-8697-87d5a5723644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:20.393085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc370560-b003-4982-8697-87d5a5723644 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:20.393574 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc370560-b003-4982-8697-87d5a5723644 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:20.394083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc370560-b003-4982-8697-87d5a5723644 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:20.395028 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc370560-b003-4982-8697-87d5a5723644 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:20.395567 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc370560-b003-4982-8697-87d5a5723644 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:20.397310 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc370560-b003-4982-8697-87d5a5723644 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:20.412875 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 446/902] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:20.419080 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c294104-ce4e-44c5-b6f2-dfa503f8e4a6 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:20.537381 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:20.546791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c294104-ce4e-44c5-b6f2-dfa503f8e4a6 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:20.587625 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e1e2b-700a-4828-99e9-dac3dd2f0936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:20.588441 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e1e2b-700a-4828-99e9-dac3dd2f0936 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:20.588778 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e1e2b-700a-4828-99e9-dac3dd2f0936 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:20.589059 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e1e2b-700a-4828-99e9-dac3dd2f0936 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:20.598227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e1e2b-700a-4828-99e9-dac3dd2f0936 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:20.599708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e1e2b-700a-4828-99e9-dac3dd2f0936 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:20.623086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-e47e1e2b-700a-4828-99e9-dac3dd2f0936 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:20.623086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:20.623086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:20.623086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:20.623086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:20.623086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:20.623086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:20.623086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:20.623086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:20.623086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:20.623086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:20.623086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:20.623086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:20.623086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:20.623086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:20.624920 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:20.624920 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:20.624920 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:20.624920 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:20.624920 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:20.624920 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:20.624920 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:20.624920 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:20.624920 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:20.624920 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:20.624920 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:20.624920 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:20.624920 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:20.624920 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:20.624920 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:20.624920 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:20.624920 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:20.625819 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:20.625819 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:20.625819 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:20.625819 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:20.625819 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:20.625819 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:20.625819 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:20.625819 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:20.625819 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 457/903] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:20.631348 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc370560-b003-4982-8697-87d5a5723644 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:20.774775 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc370560-b003-4982-8697-87d5a5723644 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:20.814948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:20.820832 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98adf2d3-0715-41bd-8d88-5bf45265bb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:20.821817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98adf2d3-0715-41bd-8d88-5bf45265bb60 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:20.821817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98adf2d3-0715-41bd-8d88-5bf45265bb60 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:20.822392 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98adf2d3-0715-41bd-8d88-5bf45265bb60 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:20.823071 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98adf2d3-0715-41bd-8d88-5bf45265bb60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:20.823511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98adf2d3-0715-41bd-8d88-5bf45265bb60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:20.824731 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98adf2d3-0715-41bd-8d88-5bf45265bb60 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:20.839812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 447/904] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:20.846537 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e1e2b-700a-4828-99e9-dac3dd2f0936 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:21.007170 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6897a-3eaf-429b-b684-33e838a9fd87 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:21.007497 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6897a-3eaf-429b-b684-33e838a9fd87 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:21.007731 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6897a-3eaf-429b-b684-33e838a9fd87 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:21.009355 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 448/905] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:59:21.016762 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09632ae3-bc97-4b36-a807-c27832376cc4 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:21.017112 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09632ae3-bc97-4b36-a807-c27832376cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:21.017465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09632ae3-bc97-4b36-a807-c27832376cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:21.017846 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e1e2b-700a-4828-99e9-dac3dd2f0936 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:21.038890 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-09632ae3-bc97-4b36-a807-c27832376cc4 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:21.040111 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-09632ae3-bc97-4b36-a807-c27832376cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:21.064677 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e75c7-f123-4c0f-9f64-d6bd2b4276fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:21.065903 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e75c7-f123-4c0f-9f64-d6bd2b4276fb None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:21.066112 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e75c7-f123-4c0f-9f64-d6bd2b4276fb None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:21.066559 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e75c7-f123-4c0f-9f64-d6bd2b4276fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:21.067531 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e75c7-f123-4c0f-9f64-d6bd2b4276fb None admin] RBAC: Adding query filter params (role.id=758069bda1404a3589771a8f43e95187, scope.project.id=b1177a85a0de4276898daf4a24ff069f, user.id=2c2e0e19880f4b048faf034dd38c7365) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:21.068003 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e75c7-f123-4c0f-9f64-d6bd2b4276fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:21.070129 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e75c7-f123-4c0f-9f64-d6bd2b4276fb None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:21.086084 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 458/906] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:59:20 2022] GET /identity/v3/role_assignments?user.id=2c2e0e19880f4b048faf034dd38c7365&scope.project.id=b1177a85a0de4276898daf4a24ff069f&role.id=758069bda1404a3589771a8f43e95187 => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:21.086084 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d774d2-5834-482d-aab6-5801afd28b5e None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:21.086084 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d774d2-5834-482d-aab6-5801afd28b5e None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:21.086084 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d774d2-5834-482d-aab6-5801afd28b5e None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:21.086084 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 459/907] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:21.087347 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6dcc1-2890-46ab-b164-c202f6129df0 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:21.087641 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6dcc1-2890-46ab-b164-c202f6129df0 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:21.087917 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6dcc1-2890-46ab-b164-c202f6129df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:21.108295 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-13d6dcc1-2890-46ab-b164-c202f6129df0 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:21.109559 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-13d6dcc1-2890-46ab-b164-c202f6129df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:21.126152 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:21.138523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 449/908] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 15:59:21.156766 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09632ae3-bc97-4b36-a807-c27832376cc4 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:21.203232 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 460/909] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 15:59:21.228236 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6dcc1-2890-46ab-b164-c202f6129df0 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:21.291258 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09632ae3-bc97-4b36-a807-c27832376cc4 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:21.320004 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:21.333536 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25bfe3-1aee-4e7d-a4a0-d21ad17659ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:21.334470 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25bfe3-1aee-4e7d-a4a0-d21ad17659ed None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:21.334760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25bfe3-1aee-4e7d-a4a0-d21ad17659ed None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:21.335054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25bfe3-1aee-4e7d-a4a0-d21ad17659ed None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:21.335777 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25bfe3-1aee-4e7d-a4a0-d21ad17659ed None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:21.337215 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25bfe3-1aee-4e7d-a4a0-d21ad17659ed None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:21.351400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-fb25bfe3-1aee-4e7d-a4a0-d21ad17659ed None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 15:59:21.351400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:21.351400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 15:59:21.351400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 15:59:21.351400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 15:59:21.351400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 15:59:21.351400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 15:59:21.351400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:21.351400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 15:59:21.351400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:21.351400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:59:21.351400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:21.351400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:21.351400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:21.351400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:21.351400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:21.352416 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:21.352416 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:21.352416 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:21.352416 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:21.352416 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:21.352416 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:21.352416 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:21.352416 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 15:59:21.352416 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 15:59:21.352416 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 15:59:21.352416 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 15:59:21.352416 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:21.352416 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:21.352416 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:21.352416 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:21.352416 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:21.353531 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:21.353531 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:21.353531 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:21.353531 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:21.353531 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:21.353531 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:21.353531 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:21.353531 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:21.353531 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:21.353531 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:21.353531 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:21.353531 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 15:59:21.353531 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:21.353531 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 15:59:21.353531 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:21.353531 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 450/910] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jun 16 15:59:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:21.362923 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25bfe3-1aee-4e7d-a4a0-d21ad17659ed None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:21.387214 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6dcc1-2890-46ab-b164-c202f6129df0 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:21.439613 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418910d-b2c4-4c52-a861-c08ed2cf4171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:21.441046 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418910d-b2c4-4c52-a861-c08ed2cf4171 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:21.441553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418910d-b2c4-4c52-a861-c08ed2cf4171 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:21.442291 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418910d-b2c4-4c52-a861-c08ed2cf4171 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:21.443554 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d418910d-b2c4-4c52-a861-c08ed2cf4171 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:21.445237 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d418910d-b2c4-4c52-a861-c08ed2cf4171 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:21.463491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-d418910d-b2c4-4c52-a861-c08ed2cf4171 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 15:59:21.463491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:21.463491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 15:59:21.463491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 15:59:21.463491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 15:59:21.463491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 15:59:21.463491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 15:59:21.463491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:21.463491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 15:59:21.463491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:21.463491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:59:21.463491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:21.463491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:21.463491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:21.463491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:21.463491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:21.463491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:21.464616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:21.464616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:21.464616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:21.464616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:21.464616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:21.464616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:21.464616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 15:59:21.464616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 15:59:21.464616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 15:59:21.464616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 15:59:21.464616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:21.464616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:21.464616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:21.464616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:21.464616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:21.464616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:21.465436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:21.465436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:21.465436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:21.465436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:21.465436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:21.465436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:21.465436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:21.465436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:21.465436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:21.465436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:21.465436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 15:59:21.465436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:21.465436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 15:59:21.465436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:21.473568 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 461/911] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 15:59:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:21.473835 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418910d-b2c4-4c52-a861-c08ed2cf4171 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:21.497715 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:21.506815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25bfe3-1aee-4e7d-a4a0-d21ad17659ed None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:21.545765 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4449c1b-2cf3-41f0-8b99-43b5c81c2fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:21.546500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4449c1b-2cf3-41f0-8b99-43b5c81c2fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:21.546782 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4449c1b-2cf3-41f0-8b99-43b5c81c2fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:21.547056 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4449c1b-2cf3-41f0-8b99-43b5c81c2fa0 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:21.547771 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4449c1b-2cf3-41f0-8b99-43b5c81c2fa0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:21.548077 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4449c1b-2cf3-41f0-8b99-43b5c81c2fa0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:21.549456 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4449c1b-2cf3-41f0-8b99-43b5c81c2fa0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:21.559913 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 451/912] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jun 16 15:59:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:21.561616 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4449c1b-2cf3-41f0-8b99-43b5c81c2fa0 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:21.626222 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418910d-b2c4-4c52-a861-c08ed2cf4171 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:21.626716 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:21.668178 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c8c4d-a1dc-4157-aa37-c79b27e9fc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:21.669060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c8c4d-a1dc-4157-aa37-c79b27e9fc3f None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:21.669363 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c8c4d-a1dc-4157-aa37-c79b27e9fc3f None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:21.669643 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c8c4d-a1dc-4157-aa37-c79b27e9fc3f None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:21.670922 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c8c4d-a1dc-4157-aa37-c79b27e9fc3f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:21.671261 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c8c4d-a1dc-4157-aa37-c79b27e9fc3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:21.672932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c8c4d-a1dc-4157-aa37-c79b27e9fc3f None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:21.682587 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 462/913] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 15:59:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:21.683637 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c8c4d-a1dc-4157-aa37-c79b27e9fc3f None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:21.693784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4449c1b-2cf3-41f0-8b99-43b5c81c2fa0 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:21.753845 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71139b36-7959-42ea-9038-f7f8dd93284d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:21.754619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71139b36-7959-42ea-9038-f7f8dd93284d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:21.754905 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71139b36-7959-42ea-9038-f7f8dd93284d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:21.755186 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71139b36-7959-42ea-9038-f7f8dd93284d None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:21.756048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71139b36-7959-42ea-9038-f7f8dd93284d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:21.756360 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71139b36-7959-42ea-9038-f7f8dd93284d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:21.757749 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71139b36-7959-42ea-9038-f7f8dd93284d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:21.769218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 452/914] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jun 16 15:59:21 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:21.771457 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71139b36-7959-42ea-9038-f7f8dd93284d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:21.795835 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:21.827435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c8c4d-a1dc-4157-aa37-c79b27e9fc3f None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:21.865176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a45d4-9024-4e71-b52e-c2720c066737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:21.866369 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a45d4-9024-4e71-b52e-c2720c066737 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:21.866883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a45d4-9024-4e71-b52e-c2720c066737 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:21.867386 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a45d4-9024-4e71-b52e-c2720c066737 None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:21.868308 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a45d4-9024-4e71-b52e-c2720c066737 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:21.868856 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a45d4-9024-4e71-b52e-c2720c066737 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:21.870577 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a45d4-9024-4e71-b52e-c2720c066737 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:21.877663 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 463/915] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 15:59:21 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:21.882947 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a45d4-9024-4e71-b52e-c2720c066737 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:21.911826 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71139b36-7959-42ea-9038-f7f8dd93284d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:21.927967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:21.961866 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b50cd-5778-4bdd-bdd0-4557dfd2e2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:21.962567 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b50cd-5778-4bdd-bdd0-4557dfd2e2f4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:21.962850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b50cd-5778-4bdd-bdd0-4557dfd2e2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:21.963126 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b50cd-5778-4bdd-bdd0-4557dfd2e2f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:21.964154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b50cd-5778-4bdd-bdd0-4557dfd2e2f4 None admin] RBAC: Adding query filter params (interface=public, service_id=443d2755aff643de976cd5489e6004f4, region_id=RegionOne) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:21.964487 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b50cd-5778-4bdd-bdd0-4557dfd2e2f4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:21.966572 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b50cd-5778-4bdd-bdd0-4557dfd2e2f4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:21.982232 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 453/916] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jun 16 15:59:21 2022] GET /identity/v3/endpoints?service_id=443d2755aff643de976cd5489e6004f4&interface=public®ion_id=RegionOne => generated 194 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:21.983239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b50cd-5778-4bdd-bdd0-4557dfd2e2f4 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:22.022678 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a45d4-9024-4e71-b52e-c2720c066737 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:22.071792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59a81b-cfd6-48ec-8de9-eef04ddb0617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:22.072834 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59a81b-cfd6-48ec-8de9-eef04ddb0617 None admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:22.073323 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59a81b-cfd6-48ec-8de9-eef04ddb0617 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:22.073802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59a81b-cfd6-48ec-8de9-eef04ddb0617 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:22.074820 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b59a81b-cfd6-48ec-8de9-eef04ddb0617 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:22.076544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b59a81b-cfd6-48ec-8de9-eef04ddb0617 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:22.086000 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:22.107133 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 464/917] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jun 16 15:59:21 2022] POST /identity/v3/endpoints => generated 346 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:59:22.112811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b50cd-5778-4bdd-bdd0-4557dfd2e2f4 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:22.161636 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2060ed-280a-4841-ac16-ca07ebe11563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:22.162320 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2060ed-280a-4841-ac16-ca07ebe11563 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:22.162773 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2060ed-280a-4841-ac16-ca07ebe11563 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:22.163066 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2060ed-280a-4841-ac16-ca07ebe11563 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:22.163767 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2060ed-280a-4841-ac16-ca07ebe11563 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:22.165250 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2060ed-280a-4841-ac16-ca07ebe11563 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:22.173448 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 454/918] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jun 16 15:59:21 2022] GET /identity/v3/services => generated 1497 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:59:22.674483 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7af54-4cf2-40fc-951c-04be93a877d1 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:22.675002 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7af54-4cf2-40fc-951c-04be93a877d1 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:22.675087 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7af54-4cf2-40fc-951c-04be93a877d1 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:22.676675 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 465/919] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:22.682738 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bcc27-6c79-498f-9af2-87719b3d0416 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:22.683030 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bcc27-6c79-498f-9af2-87719b3d0416 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:22.683317 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bcc27-6c79-498f-9af2-87719b3d0416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:22.700854 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-055bcc27-6c79-498f-9af2-87719b3d0416 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:22.702659 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-055bcc27-6c79-498f-9af2-87719b3d0416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:22.736206 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:22.777236 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 455/920] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:22.801701 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7af54-4cf2-40fc-951c-04be93a877d1 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:22.917159 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7af54-4cf2-40fc-951c-04be93a877d1 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:22.959383 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd3341-5312-4ec0-8fad-b7630e43b529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:22.960526 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd3341-5312-4ec0-8fad-b7630e43b529 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:22.960992 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd3341-5312-4ec0-8fad-b7630e43b529 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:22.961446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd3341-5312-4ec0-8fad-b7630e43b529 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:22.962329 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd3341-5312-4ec0-8fad-b7630e43b529 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:22.963935 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd3341-5312-4ec0-8fad-b7630e43b529 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:22.977264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-85fd3341-5312-4ec0-8fad-b7630e43b529 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 15:59:22.977264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:22.977264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 15:59:22.977264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 15:59:22.977264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 15:59:22.977264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 15:59:22.977264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 15:59:22.977264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:22.977264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 15:59:22.977264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:22.977264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:59:22.977264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:22.977264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:22.977264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:22.977264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:22.977264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:22.977264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:22.978346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:22.978346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:22.978346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:22.978346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:22.978346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:22.978346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:22.978346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 15:59:22.978346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 15:59:22.978346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 15:59:22.978346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 15:59:22.978346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:22.978346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:22.978346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:22.978346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:22.978346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:22.978346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:22.979169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:22.979169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:22.979169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:22.979169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:22.979169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:22.979169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:22.979169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:22.979169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:22.979169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:22.979169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:22.979169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 15:59:22.979169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:22.979169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 15:59:22.979169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:22.981665 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 466/921] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jun 16 15:59:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:22.988067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055bcc27-6c79-498f-9af2-87719b3d0416 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:23.104212 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055bcc27-6c79-498f-9af2-87719b3d0416 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:23.144468 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4d69e-46a3-4912-a87f-8f2bc6d010da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:23.145395 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4d69e-46a3-4912-a87f-8f2bc6d010da None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:23.145854 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4d69e-46a3-4912-a87f-8f2bc6d010da None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:23.146296 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4d69e-46a3-4912-a87f-8f2bc6d010da None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:23.147153 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e4d69e-46a3-4912-a87f-8f2bc6d010da None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:23.147604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e4d69e-46a3-4912-a87f-8f2bc6d010da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:23.149149 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e4d69e-46a3-4912-a87f-8f2bc6d010da None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:23.154808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 456/922] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jun 16 15:59:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:23.160274 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd3341-5312-4ec0-8fad-b7630e43b529 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:23.226297 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:23.274081 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd3341-5312-4ec0-8fad-b7630e43b529 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:23.313621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730070d-a2a0-4ecc-a64e-990d23ace65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:23.314592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730070d-a2a0-4ecc-a64e-990d23ace65b None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:23.315051 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730070d-a2a0-4ecc-a64e-990d23ace65b None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:23.315476 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730070d-a2a0-4ecc-a64e-990d23ace65b None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:23.316379 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730070d-a2a0-4ecc-a64e-990d23ace65b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:23.316842 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730070d-a2a0-4ecc-a64e-990d23ace65b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:23.318438 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730070d-a2a0-4ecc-a64e-990d23ace65b None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:23.324831 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 467/923] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jun 16 15:59:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:23.330788 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4d69e-46a3-4912-a87f-8f2bc6d010da None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:23.371124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:23.416144 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafde0bf-03c2-41f6-aed6-18404d731ece None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:23.416430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafde0bf-03c2-41f6-aed6-18404d731ece None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:23.416710 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafde0bf-03c2-41f6-aed6-18404d731ece None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:23.422067 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 468/924] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:59:23.425494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e31efa-fc97-4805-9d34-3304f175aa74 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:23.425783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e31efa-fc97-4805-9d34-3304f175aa74 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:23.426096 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e31efa-fc97-4805-9d34-3304f175aa74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:23.449858 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-38e31efa-fc97-4805-9d34-3304f175aa74 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:23.451888 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-38e31efa-fc97-4805-9d34-3304f175aa74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:23.472595 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4d69e-46a3-4912-a87f-8f2bc6d010da None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:23.519630 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115ce40-a3a3-4505-a95e-d79760b34abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:23.520768 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115ce40-a3a3-4505-a95e-d79760b34abc None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:23.521315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115ce40-a3a3-4505-a95e-d79760b34abc None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:23.521832 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115ce40-a3a3-4505-a95e-d79760b34abc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:23.522936 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115ce40-a3a3-4505-a95e-d79760b34abc None admin] RBAC: Adding query filter params (interface=public, service_id=079a7542c1374120ba40e2b0d3f58513, region_id=RegionOne) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:23.523595 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115ce40-a3a3-4505-a95e-d79760b34abc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:23.526535 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115ce40-a3a3-4505-a95e-d79760b34abc None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:23.536718 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 457/925] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jun 16 15:59:23 2022] GET /identity/v3/endpoints?service_id=079a7542c1374120ba40e2b0d3f58513&interface=public®ion_id=RegionOne => generated 194 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:23.546043 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115ce40-a3a3-4505-a95e-d79760b34abc None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:23.556419 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 469/926] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:23.579131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e31efa-fc97-4805-9d34-3304f175aa74 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:23.641796 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:23.647463 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:23.656867 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115ce40-a3a3-4505-a95e-d79760b34abc None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:23.695315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2948c-c484-444e-a11b-ae2a33f31080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:23.696345 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2948c-c484-444e-a11b-ae2a33f31080 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:23.696737 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2948c-c484-444e-a11b-ae2a33f31080 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:23.697259 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2948c-c484-444e-a11b-ae2a33f31080 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:23.697992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2948c-c484-444e-a11b-ae2a33f31080 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:23.699383 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2948c-c484-444e-a11b-ae2a33f31080 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:23.699690 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e31efa-fc97-4805-9d34-3304f175aa74 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:23.710004 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 458/927] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jun 16 15:59:23 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:59:23.750302 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65f9b6-161b-417b-8ff4-20db8599ce41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:23.751621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65f9b6-161b-417b-8ff4-20db8599ce41 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:23.752115 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65f9b6-161b-417b-8ff4-20db8599ce41 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:23.752589 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65f9b6-161b-417b-8ff4-20db8599ce41 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:23.767262 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b65f9b6-161b-417b-8ff4-20db8599ce41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:23.777553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-9b65f9b6-161b-417b-8ff4-20db8599ce41 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 15:59:23.777553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:23.777553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:23.777553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:23.777553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:23.777553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:23.777553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:23.777553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:23.777553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:23.777553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:23.777553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:23.777553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:23.777553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:23.777553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:23.777553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:23.777553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:23.778706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:23.778706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:23.778706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:23.778706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:23.778706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:23.778706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:23.778706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:23.778706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:23.778706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:23.778706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:23.778706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:23.778706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:23.778706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:23.778706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:23.778706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:23.778706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:23.780480 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:23.780480 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:23.780480 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:23.780480 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:23.780480 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:23.780480 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:23.780480 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:23.780480 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:23.780480 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:23.780480 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:23.780480 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 15:59:23.780480 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:23.784364 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 470/928] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jun 16 15:59:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:23.793891 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2948c-c484-444e-a11b-ae2a33f31080 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:23.923106 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2948c-c484-444e-a11b-ae2a33f31080 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:23.962611 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f2759-468c-47ac-a720-ed18cc07f70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:23.963300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f2759-468c-47ac-a720-ed18cc07f70a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:23.963590 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f2759-468c-47ac-a720-ed18cc07f70a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:23.963869 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f2759-468c-47ac-a720-ed18cc07f70a None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:23.964634 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f2759-468c-47ac-a720-ed18cc07f70a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:23.964945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f2759-468c-47ac-a720-ed18cc07f70a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:23.966380 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f2759-468c-47ac-a720-ed18cc07f70a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:23.977598 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 459/929] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jun 16 15:59:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:23.983621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65f9b6-161b-417b-8ff4-20db8599ce41 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:24.096849 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65f9b6-161b-417b-8ff4-20db8599ce41 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:24.111802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:24.141506 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46554ab9-86b6-46d7-947c-9aef6150108c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:24.142487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46554ab9-86b6-46d7-947c-9aef6150108c None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:24.142769 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46554ab9-86b6-46d7-947c-9aef6150108c None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:24.143086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46554ab9-86b6-46d7-947c-9aef6150108c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:24.155610 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46554ab9-86b6-46d7-947c-9aef6150108c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:24.157116 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46554ab9-86b6-46d7-947c-9aef6150108c None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:24.176110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-46554ab9-86b6-46d7-947c-9aef6150108c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:24.176110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:24.176110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:24.176110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:24.176110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:24.176110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:24.176110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:24.176110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:24.176110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:24.176110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:24.176110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:24.176110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:24.176110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:24.176110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:24.176110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:24.177056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:24.177056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:24.177056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:24.177056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:24.177056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:24.177056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:24.177056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:24.177056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:24.177056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:24.177056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:24.177056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:24.177056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:24.177056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:24.177056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:24.177056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:24.177056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:24.177056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:24.178020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:24.178020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:24.178020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:24.178020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:24.178834 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 471/930] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:24.188657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f2759-468c-47ac-a720-ed18cc07f70a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:24.220869 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:24.320923 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f2759-468c-47ac-a720-ed18cc07f70a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:24.359418 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c256bf-34f3-4a2d-b9b7-7a85b3faa366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:24.360444 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c256bf-34f3-4a2d-b9b7-7a85b3faa366 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:24.360954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c256bf-34f3-4a2d-b9b7-7a85b3faa366 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:24.361435 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c256bf-34f3-4a2d-b9b7-7a85b3faa366 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:24.362404 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c256bf-34f3-4a2d-b9b7-7a85b3faa366 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:24.362918 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c256bf-34f3-4a2d-b9b7-7a85b3faa366 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:24.364491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c256bf-34f3-4a2d-b9b7-7a85b3faa366 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:24.379629 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 460/931] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:24.385961 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46554ab9-86b6-46d7-947c-9aef6150108c None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:24.455801 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346bebe-3670-46b7-9680-2a9241547471 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:24.456453 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346bebe-3670-46b7-9680-2a9241547471 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:24.456984 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346bebe-3670-46b7-9680-2a9241547471 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:24.462592 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 461/932] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:59:24.464887 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fa6d5-44fd-41b1-a079-7bb0bd89cee7 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:24.465408 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fa6d5-44fd-41b1-a079-7bb0bd89cee7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:24.465894 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fa6d5-44fd-41b1-a079-7bb0bd89cee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:24.485073 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d14fa6d5-44fd-41b1-a079-7bb0bd89cee7 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:24.486703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d14fa6d5-44fd-41b1-a079-7bb0bd89cee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:24.519489 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46554ab9-86b6-46d7-947c-9aef6150108c None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:24.563205 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3d5f6-a47f-49e2-8006-0dc9804b4228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:24.563841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3d5f6-a47f-49e2-8006-0dc9804b4228 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:24.564122 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3d5f6-a47f-49e2-8006-0dc9804b4228 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:24.564428 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3d5f6-a47f-49e2-8006-0dc9804b4228 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:24.576481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 462/933] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 15:59:24.576672 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e3d5f6-a47f-49e2-8006-0dc9804b4228 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:24.576767 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce05fac-acdd-4a15-b53e-91ffab677f8a None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:24.576767 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce05fac-acdd-4a15-b53e-91ffab677f8a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:24.576941 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e3d5f6-a47f-49e2-8006-0dc9804b4228 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:24.577041 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce05fac-acdd-4a15-b53e-91ffab677f8a None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:24.577041 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 463/934] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:24.580809 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3176ce-8a4a-4347-8e05-ab8a7715f68d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:24.581107 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3176ce-8a4a-4347-8e05-ab8a7715f68d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:24.581393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3176ce-8a4a-4347-8e05-ab8a7715f68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:24.595042 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-78e3d5f6-a47f-49e2-8006-0dc9804b4228 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 15:59:24.595042 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:24.595042 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:24.595042 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:24.595042 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:24.595042 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:24.595042 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:24.595042 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:24.595042 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:24.595042 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:24.595042 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:24.595042 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:24.595042 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:24.595042 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:24.595042 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:24.595042 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:24.596028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:24.596028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:24.596028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:24.596028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:24.596028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:24.596028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:24.596028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:24.596028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:24.596028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:24.596028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:24.596028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:24.596028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:24.596028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:24.596028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:24.596028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:24.596028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:24.596028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:24.596028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:24.596968 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:24.596968 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:24.596968 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:24.596968 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:24.596968 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:24.596968 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:24.596968 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:24.596968 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:24.596968 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 15:59:24.596968 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:24.599358 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ab3176ce-8a4a-4347-8e05-ab8a7715f68d None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:24.599808 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 472/935] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jun 16 15:59:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:24.600569 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ab3176ce-8a4a-4347-8e05-ab8a7715f68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:24.604072 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3d5f6-a47f-49e2-8006-0dc9804b4228 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:24.609766 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:24.635881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:24.684128 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 464/936] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 15:59:24.684128 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3176ce-8a4a-4347-8e05-ab8a7715f68d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:24.735343 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3d5f6-a47f-49e2-8006-0dc9804b4228 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:24.772821 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbcd50e-3179-4dce-a383-9480688efef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:24.773800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbcd50e-3179-4dce-a383-9480688efef0 None admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:24.774115 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbcd50e-3179-4dce-a383-9480688efef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:24.774396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbcd50e-3179-4dce-a383-9480688efef0 None admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:24.775102 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbcd50e-3179-4dce-a383-9480688efef0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:24.776558 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbcd50e-3179-4dce-a383-9480688efef0 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:24.777971 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: WARNING keystone.server.flask.common [None req-acbcd50e-3179-4dce-a383-9480688efef0 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 16 15:59:24.800276 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.sql.core [None req-acbcd50e-3179-4dce-a383-9480688efef0 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 15:59:24.800276 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 15:59:24.800276 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [parameters: {'id': 'd27f3d66f0684b76b237a44f36b60d1a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 15:59:24.800276 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80439) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 15:59:24.802912 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3176ce-8a4a-4347-8e05-ab8a7715f68d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:24.820080 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-acbcd50e-3179-4dce-a383-9480688efef0 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d27f3d66f0684b76b237a44f36b60d1a.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d27f3d66f0684b76b237a44f36b60d1a. Jun 16 15:59:24.820080 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:24.820080 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 15:59:24.820080 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 15:59:24.820080 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 15:59:24.820080 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 15:59:24.820080 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 15:59:24.820080 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 15:59:24.820080 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 15:59:24.820080 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application conn.query(q) Jun 16 15:59:24.820080 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 15:59:24.820080 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 15:59:24.820080 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 15:59:24.820949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application result.read() Jun 16 15:59:24.820949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 15:59:24.820949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 15:59:24.820949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 15:59:24.820949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 15:59:24.820949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 15:59:24.820949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 15:59:24.820949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 15:59:24.820949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 15:59:24.820949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 15:59:24.820949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:24.820949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 15:59:24.820949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:24.820949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:24.820949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 15:59:24.820949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 15:59:24.820949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 16 15:59:24.820949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 16 15:59:24.821895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 15:59:24.821895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application next(self.gen) Jun 16 15:59:24.821895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 15:59:24.821895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application yield resource Jun 16 15:59:24.821895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 15:59:24.821895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application next(self.gen) Jun 16 15:59:24.821895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 15:59:24.821895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 15:59:24.821895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 15:59:24.821895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 15:59:24.821895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 15:59:24.821895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise self.value Jun 16 15:59:24.821895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 15:59:24.821895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 15:59:24.821895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 15:59:24.821895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application session.commit() Jun 16 15:59:24.821895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 15:59:24.822843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 15:59:24.822843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 15:59:24.822843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 15:59:24.822843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 15:59:24.822843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self.session.flush() Jun 16 15:59:24.822843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 15:59:24.822843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 15:59:24.822843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 15:59:24.822843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 15:59:24.822843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 15:59:24.822843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application compat.raise_( Jun 16 15:59:24.822843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 15:59:24.822843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception Jun 16 15:59:24.822843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 15:59:24.822843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 15:59:24.822843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 15:59:24.822843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 15:59:24.823739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 15:59:24.823739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 15:59:24.823739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 15:59:24.823739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 15:59:24.823739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 15:59:24.823739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 15:59:24.823739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 15:59:24.823739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 15:59:24.823739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 15:59:24.823739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 15:59:24.823739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 15:59:24.823739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 15:59:24.823739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 15:59:24.823739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 15:59:24.823739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 15:59:24.824533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 15:59:24.824533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 15:59:24.824533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception Jun 16 15:59:24.824533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 15:59:24.824533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 15:59:24.824533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 15:59:24.824533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 15:59:24.824533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 15:59:24.824533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 15:59:24.824533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 15:59:24.824533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application conn.query(q) Jun 16 15:59:24.824533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 15:59:24.824533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 15:59:24.824533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 15:59:24.824533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application result.read() Jun 16 15:59:24.824533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 15:59:24.824533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 15:59:24.825409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 15:59:24.825409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 15:59:24.825409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 15:59:24.825409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 15:59:24.825409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 15:59:24.825409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 15:59:24.825409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 15:59:24.825409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 15:59:24.825409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [parameters: {'id': 'd27f3d66f0684b76b237a44f36b60d1a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 15:59:24.825409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 15:59:24.825409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:24.825409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:59:24.825409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:24.825409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:24.825409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 16 15:59:24.827844 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 16 15:59:24.827844 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 15:59:24.827844 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 15:59:24.827844 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 16 15:59:24.827844 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:24.827844 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:59:24.827844 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:24.827844 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:24.827844 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:24.827844 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:24.827844 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:24.827844 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:24.827844 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:24.827844 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:24.827844 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:24.827844 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:24.827844 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:24.828963 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:24.828963 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 16 15:59:24.828963 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 16 15:59:24.828963 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:24.828963 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:24.828963 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 16 15:59:24.828963 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 16 15:59:24.828963 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d27f3d66f0684b76b237a44f36b60d1a. Jun 16 15:59:24.828963 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:24.828963 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 473/937] 10.222.0.17 () {60 vars in 1140 bytes} [Thu Jun 16 15:59:24 2022] POST /identity/v3/projects => generated 262 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:24.828963 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbcd50e-3179-4dce-a383-9480688efef0 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:24.850461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b33b82-23cd-4642-8bcf-a2055cf5bcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:24.851239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b33b82-23cd-4642-8bcf-a2055cf5bcad None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:24.851588 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b33b82-23cd-4642-8bcf-a2055cf5bcad None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:24.851930 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b33b82-23cd-4642-8bcf-a2055cf5bcad None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:24.852840 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b33b82-23cd-4642-8bcf-a2055cf5bcad None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:24.853219 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b33b82-23cd-4642-8bcf-a2055cf5bcad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:24.854774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b33b82-23cd-4642-8bcf-a2055cf5bcad None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:24.869772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:24.874147 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 465/938] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jun 16 15:59:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:24.875063 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b33b82-23cd-4642-8bcf-a2055cf5bcad None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:24.928088 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:24.954972 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbcd50e-3179-4dce-a383-9480688efef0 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:24.990969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b33b82-23cd-4642-8bcf-a2055cf5bcad None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:24.992717 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70be9c-1a2a-4b51-9d48-199470d3a36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:24.993779 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70be9c-1a2a-4b51-9d48-199470d3a36b None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:24.994093 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70be9c-1a2a-4b51-9d48-199470d3a36b None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:24.994370 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70be9c-1a2a-4b51-9d48-199470d3a36b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:24.995021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb70be9c-1a2a-4b51-9d48-199470d3a36b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:24.996601 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb70be9c-1a2a-4b51-9d48-199470d3a36b None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:25.010487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-fb70be9c-1a2a-4b51-9d48-199470d3a36b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 15:59:25.010487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:25.010487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 15:59:25.010487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 15:59:25.010487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 15:59:25.010487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 15:59:25.010487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 15:59:25.010487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:25.010487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 15:59:25.010487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:25.010487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:59:25.010487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:25.010487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:25.010487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:25.010487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:25.010487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:25.011484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:25.011484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:25.011484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:25.011484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:25.011484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:25.011484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:25.011484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:25.011484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 15:59:25.011484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 15:59:25.011484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 15:59:25.011484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 15:59:25.011484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:25.011484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:25.011484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:25.011484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:25.011484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:25.012333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:25.012333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:25.012333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:25.012333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:25.012333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:25.012333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:25.012333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:25.012333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:25.012333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:25.012333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:25.012333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:25.012333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 15:59:25.012333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:25.012333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 15:59:25.012333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:25.012333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 474/939] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jun 16 15:59:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:25.013176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70be9c-1a2a-4b51-9d48-199470d3a36b None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:25.035907 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c0573-994f-4f7b-b033-0ecb2b703161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:25.036706 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c0573-994f-4f7b-b033-0ecb2b703161 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:25.036993 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c0573-994f-4f7b-b033-0ecb2b703161 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:25.037275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c0573-994f-4f7b-b033-0ecb2b703161 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:25.044446 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c0573-994f-4f7b-b033-0ecb2b703161 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:25.046212 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c0573-994f-4f7b-b033-0ecb2b703161 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:25.060108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-c34c0573-994f-4f7b-b033-0ecb2b703161 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:25.060108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:25.060108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:25.060108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:25.060108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:25.060108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:25.060108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:25.060108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:25.060108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:25.060108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:25.060108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:25.060108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:25.060108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:25.060108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:25.060108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:25.062055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:25.062055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:25.062055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:25.062055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:25.062055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:25.062055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:25.062055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:25.062055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:25.062055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:25.062055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:25.062055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:25.062055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:25.062055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:25.062055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:25.062055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:25.062055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:25.062055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:25.063046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:25.063046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:25.063046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:25.063046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:25.063046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:25.063046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:25.063046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:25.063046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:25.063046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 466/940] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:25.063682 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c0573-994f-4f7b-b033-0ecb2b703161 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:25.136881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70be9c-1a2a-4b51-9d48-199470d3a36b None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:25.140184 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:25.177458 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e28af8-1e42-4c0c-826a-2d4c1fbc9707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:25.178284 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e28af8-1e42-4c0c-826a-2d4c1fbc9707 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:25.178564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e28af8-1e42-4c0c-826a-2d4c1fbc9707 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:25.178949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e28af8-1e42-4c0c-826a-2d4c1fbc9707 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:25.182321 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c0573-994f-4f7b-b033-0ecb2b703161 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:25.192998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e28af8-1e42-4c0c-826a-2d4c1fbc9707 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:25.194615 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e28af8-1e42-4c0c-826a-2d4c1fbc9707 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:25.202985 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:25.216078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-74e28af8-1e42-4c0c-826a-2d4c1fbc9707 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:25.216078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:25.216078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:25.216078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:25.216078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:25.216078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:25.216078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:25.216078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:25.216078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:25.216078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:25.216078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:25.216078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:25.216078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:25.216078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:25.216078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:25.217248 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:25.217248 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:25.217248 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:25.217248 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:25.217248 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:25.217248 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:25.217248 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:25.217248 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:25.217248 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:25.217248 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:25.217248 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:25.217248 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:25.217248 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:25.217248 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:25.217248 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:25.217248 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:25.217248 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:25.218215 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:25.218215 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:25.218215 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:25.218215 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:25.218215 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 475/941] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:25.218992 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e28af8-1e42-4c0c-826a-2d4c1fbc9707 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:25.224813 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda6b0d-9b5e-4d55-b2ec-62abf3c20a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:25.225524 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda6b0d-9b5e-4d55-b2ec-62abf3c20a55 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:25.225813 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda6b0d-9b5e-4d55-b2ec-62abf3c20a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:25.226125 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda6b0d-9b5e-4d55-b2ec-62abf3c20a55 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:25.226844 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda6b0d-9b5e-4d55-b2ec-62abf3c20a55 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:25.227153 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda6b0d-9b5e-4d55-b2ec-62abf3c20a55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:25.228523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda6b0d-9b5e-4d55-b2ec-62abf3c20a55 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:25.238766 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 467/942] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jun 16 15:59:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:25.238766 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda6b0d-9b5e-4d55-b2ec-62abf3c20a55 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:25.346703 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e28af8-1e42-4c0c-826a-2d4c1fbc9707 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:25.368195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda6b0d-9b5e-4d55-b2ec-62abf3c20a55 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:25.389555 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7035e-b71b-4bd5-ba34-bc96b5b4969d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:25.390196 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7035e-b71b-4bd5-ba34-bc96b5b4969d None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:25.390489 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7035e-b71b-4bd5-ba34-bc96b5b4969d None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:25.390746 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7035e-b71b-4bd5-ba34-bc96b5b4969d None admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:25.391508 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db7035e-b71b-4bd5-ba34-bc96b5b4969d None admin] RBAC: Adding query filter params (name=service) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:25.391813 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db7035e-b71b-4bd5-ba34-bc96b5b4969d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:25.393257 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db7035e-b71b-4bd5-ba34-bc96b5b4969d None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:25.416040 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:25.428066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 476/943] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jun 16 15:59:25 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:25.429122 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7035e-b71b-4bd5-ba34-bc96b5b4969d None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:25.445765 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ccd0b1-a45d-42ed-a21a-1c46cd885858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:25.448141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ccd0b1-a45d-42ed-a21a-1c46cd885858 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:25.448738 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ccd0b1-a45d-42ed-a21a-1c46cd885858 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:25.450193 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ccd0b1-a45d-42ed-a21a-1c46cd885858 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:25.451870 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ccd0b1-a45d-42ed-a21a-1c46cd885858 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:25.452804 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ccd0b1-a45d-42ed-a21a-1c46cd885858 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:25.455196 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ccd0b1-a45d-42ed-a21a-1c46cd885858 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:25.478631 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 468/944] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:25.481103 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ccd0b1-a45d-42ed-a21a-1c46cd885858 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:25.539552 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:25.615757 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7035e-b71b-4bd5-ba34-bc96b5b4969d None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:25.639640 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ccd0b1-a45d-42ed-a21a-1c46cd885858 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:25.668792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af2b1fa-a4a0-49a6-9ac5-1bbf558da53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:25.669558 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af2b1fa-a4a0-49a6-9ac5-1bbf558da53c None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:25.669807 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af2b1fa-a4a0-49a6-9ac5-1bbf558da53c None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:25.670659 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af2b1fa-a4a0-49a6-9ac5-1bbf558da53c None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:25.671423 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af2b1fa-a4a0-49a6-9ac5-1bbf558da53c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:25.671772 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af2b1fa-a4a0-49a6-9ac5-1bbf558da53c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:25.673137 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af2b1fa-a4a0-49a6-9ac5-1bbf558da53c None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:25.678923 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 477/945] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jun 16 15:59:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:25.684527 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af2b1fa-a4a0-49a6-9ac5-1bbf558da53c None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:25.695295 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55312f7-8291-42e3-bdaa-901f0b87add7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:25.696814 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55312f7-8291-42e3-bdaa-901f0b87add7 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:25.697357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55312f7-8291-42e3-bdaa-901f0b87add7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:25.698211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55312f7-8291-42e3-bdaa-901f0b87add7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:25.711677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55312f7-8291-42e3-bdaa-901f0b87add7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:25.714792 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55312f7-8291-42e3-bdaa-901f0b87add7 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:25.732072 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-c55312f7-8291-42e3-bdaa-901f0b87add7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:25.732072 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:25.732072 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:25.732072 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:25.732072 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:25.732072 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:25.732072 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:25.732072 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:25.732072 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:25.732072 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:25.732072 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:25.732072 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:25.732072 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:25.732072 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:25.732072 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:25.733587 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:25.733587 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:25.733587 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:25.733587 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:25.733587 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:25.733587 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:25.733587 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:25.733587 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:25.733587 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:25.733587 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:25.733587 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:25.733587 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:25.733587 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:25.733587 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:25.733587 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:25.733587 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:25.733587 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:25.734526 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:25.734526 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:25.734526 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:25.734526 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:25.734526 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:25.734526 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:25.734526 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:25.734526 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:25.740106 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 469/946] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:25.741331 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55312f7-8291-42e3-bdaa-901f0b87add7 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:25.819591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af2b1fa-a4a0-49a6-9ac5-1bbf558da53c None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:25.847082 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:25.862002 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bec0a-7667-43e2-82be-967aa2534b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:25.862680 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bec0a-7667-43e2-82be-967aa2534b58 None admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:25.862964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bec0a-7667-43e2-82be-967aa2534b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:25.863240 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bec0a-7667-43e2-82be-967aa2534b58 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:25.864107 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818bec0a-7667-43e2-82be-967aa2534b58 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:25.865372 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818bec0a-7667-43e2-82be-967aa2534b58 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:25.874338 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:25.885655 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55312f7-8291-42e3-bdaa-901f0b87add7 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:25.895854 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 478/947] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jun 16 15:59:25 2022] POST /identity/v3/endpoints => generated 347 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:59:25.940974 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd11ae7-4f9c-4c15-bdc1-097c62364b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:25.942073 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd11ae7-4f9c-4c15-bdc1-097c62364b35 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:25.942548 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd11ae7-4f9c-4c15-bdc1-097c62364b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:25.942966 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd11ae7-4f9c-4c15-bdc1-097c62364b35 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:25.944125 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd11ae7-4f9c-4c15-bdc1-097c62364b35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:25.944693 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd11ae7-4f9c-4c15-bdc1-097c62364b35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:25.946490 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd11ae7-4f9c-4c15-bdc1-097c62364b35 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:25.969503 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 470/948] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:25.971385 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bec0a-7667-43e2-82be-967aa2534b58 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:26.035312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d1bcf-2f4d-4121-ae76-ce944e78e4b0 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:26.035775 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d1bcf-2f4d-4121-ae76-ce944e78e4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:26.036159 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d1bcf-2f4d-4121-ae76-ce944e78e4b0 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:26.038038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 471/949] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:26.043777 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e651033-5eea-4134-8a58-b7e5157c1eee None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:26.044170 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e651033-5eea-4134-8a58-b7e5157c1eee None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:26.044554 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e651033-5eea-4134-8a58-b7e5157c1eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:26.070678 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8e651033-5eea-4134-8a58-b7e5157c1eee None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:26.072274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8e651033-5eea-4134-8a58-b7e5157c1eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:26.103006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bec0a-7667-43e2-82be-967aa2534b58 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:26.143442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ded7f-ae3e-4221-83a8-530b12ec6902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:26.145213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ded7f-ae3e-4221-83a8-530b12ec6902 None admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:26.145213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ded7f-ae3e-4221-83a8-530b12ec6902 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:26.145213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ded7f-ae3e-4221-83a8-530b12ec6902 None admin] PATH_INFO: `/v3/projects/b1177a85a0de4276898daf4a24ff069f/users/2c2e0e19880f4b048faf034dd38c7365/roles/758069bda1404a3589771a8f43e95187` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:26.147575 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ded7f-ae3e-4221-83a8-530b12ec6902 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1177a85a0de4276898daf4a24ff069f, user_id=2c2e0e19880f4b048faf034dd38c7365, role_id=758069bda1404a3589771a8f43e95187)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:26.149077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ded7f-ae3e-4221-83a8-530b12ec6902 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:26.165675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 472/950] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:26.166799 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 479/951] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 15:59:25 2022] PUT /identity/v3/projects/b1177a85a0de4276898daf4a24ff069f/users/2c2e0e19880f4b048faf034dd38c7365/roles/758069bda1404a3589771a8f43e95187 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:59:26.189483 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e651033-5eea-4134-8a58-b7e5157c1eee None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:26.223363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:26.311569 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e651033-5eea-4134-8a58-b7e5157c1eee None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:26.353712 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd0238-7d7a-4fad-8367-2eedae31084c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:26.354584 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd0238-7d7a-4fad-8367-2eedae31084c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:26.354869 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd0238-7d7a-4fad-8367-2eedae31084c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:26.355144 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd0238-7d7a-4fad-8367-2eedae31084c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:26.355867 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd0238-7d7a-4fad-8367-2eedae31084c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:26.357822 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd0238-7d7a-4fad-8367-2eedae31084c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:26.372807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-e2bd0238-7d7a-4fad-8367-2eedae31084c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 15:59:26.372807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:26.372807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 15:59:26.372807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 15:59:26.372807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 15:59:26.372807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 15:59:26.372807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 15:59:26.372807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:26.372807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 15:59:26.372807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:26.372807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:59:26.372807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:26.372807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:26.372807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:26.372807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:26.372807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:26.372807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:26.373838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:26.373838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:26.373838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:26.373838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:26.373838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:26.373838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:26.373838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 15:59:26.373838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 15:59:26.373838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 15:59:26.373838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 15:59:26.373838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:26.373838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:26.373838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:26.373838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:26.373838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:26.373838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:26.374888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:26.374888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:26.374888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:26.374888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:26.374888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:26.374888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:26.374888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:26.374888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:26.374888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:26.374888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:26.374888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 15:59:26.374888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:26.374888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 15:59:26.374888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:26.374888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 473/952] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jun 16 15:59:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:26.380639 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ded7f-ae3e-4221-83a8-530b12ec6902 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:26.425799 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:26.491919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ded7f-ae3e-4221-83a8-530b12ec6902 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:26.533302 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1efbaa-aa61-4d66-bbf8-80ea6d823979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:26.534009 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1efbaa-aa61-4d66-bbf8-80ea6d823979 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:26.534295 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1efbaa-aa61-4d66-bbf8-80ea6d823979 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:26.534573 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1efbaa-aa61-4d66-bbf8-80ea6d823979 None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:26.535287 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1efbaa-aa61-4d66-bbf8-80ea6d823979 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:26.535594 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1efbaa-aa61-4d66-bbf8-80ea6d823979 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:26.536977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1efbaa-aa61-4d66-bbf8-80ea6d823979 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:26.542701 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 480/953] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jun 16 15:59:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:26.548760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd0238-7d7a-4fad-8367-2eedae31084c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:26.659828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:26.668253 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd0238-7d7a-4fad-8367-2eedae31084c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:26.712434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c005364-d32f-47ed-851d-0117eed21cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:26.713503 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c005364-d32f-47ed-851d-0117eed21cee None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:26.714054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c005364-d32f-47ed-851d-0117eed21cee None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:26.714530 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c005364-d32f-47ed-851d-0117eed21cee None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:26.715470 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c005364-d32f-47ed-851d-0117eed21cee None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:26.716027 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c005364-d32f-47ed-851d-0117eed21cee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:26.718255 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c005364-d32f-47ed-851d-0117eed21cee None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:26.724953 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 474/954] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jun 16 15:59:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:26.730593 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1efbaa-aa61-4d66-bbf8-80ea6d823979 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:26.859326 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1efbaa-aa61-4d66-bbf8-80ea6d823979 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:26.860910 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:26.902298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab3439-37c0-4933-ac8e-f0a58e91224d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:26.902977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab3439-37c0-4933-ac8e-f0a58e91224d None admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:26.903262 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab3439-37c0-4933-ac8e-f0a58e91224d None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:26.903542 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab3439-37c0-4933-ac8e-f0a58e91224d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:26.904335 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab3439-37c0-4933-ac8e-f0a58e91224d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:26.905731 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab3439-37c0-4933-ac8e-f0a58e91224d None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:26.926778 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 481/955] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jun 16 15:59:26 2022] POST /identity/v3/endpoints => generated 349 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:59:27.822039 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d19123-259b-4260-9139-d302f5a362dd None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:27.823424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d19123-259b-4260-9139-d302f5a362dd None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:27.824162 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d19123-259b-4260-9139-d302f5a362dd None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:27.827334 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 475/956] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:27.833200 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4795abb-80ac-458f-bf6e-5929de175e95 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:27.833502 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4795abb-80ac-458f-bf6e-5929de175e95 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:27.833786 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4795abb-80ac-458f-bf6e-5929de175e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:27.852259 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-f4795abb-80ac-458f-bf6e-5929de175e95 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:27.853789 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-f4795abb-80ac-458f-bf6e-5929de175e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:27.939676 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 482/957] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:27.973780 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d19123-259b-4260-9139-d302f5a362dd None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:28.100727 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d19123-259b-4260-9139-d302f5a362dd None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:28.141444 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e1f49-7a05-4337-bf6b-13cfb0353840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:28.142454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e1f49-7a05-4337-bf6b-13cfb0353840 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:28.142883 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e1f49-7a05-4337-bf6b-13cfb0353840 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:28.143299 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e1f49-7a05-4337-bf6b-13cfb0353840 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:28.144248 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e1f49-7a05-4337-bf6b-13cfb0353840 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:28.146023 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e1f49-7a05-4337-bf6b-13cfb0353840 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:28.147587 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: WARNING keystone.server.flask.common [None req-298e1f49-7a05-4337-bf6b-13cfb0353840 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 16 15:59:28.161866 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.sql.core [None req-298e1f49-7a05-4337-bf6b-13cfb0353840 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 15:59:28.161866 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 15:59:28.161866 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [parameters: {'user_id': 'fd8485e411634625893ad36ecfc25a61', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 15:59:28.161866 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80440) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 15:59:28.171473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-298e1f49-7a05-4337-bf6b-13cfb0353840 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 15:59:28.171473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:28.171473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 15:59:28.171473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 15:59:28.171473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 15:59:28.171473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 15:59:28.171473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 15:59:28.171473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 15:59:28.171473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 15:59:28.171473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application conn.query(q) Jun 16 15:59:28.171473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 15:59:28.171473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 15:59:28.171473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 15:59:28.171473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application result.read() Jun 16 15:59:28.171473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 15:59:28.172583 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 15:59:28.172583 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 15:59:28.172583 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 15:59:28.172583 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 15:59:28.172583 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 15:59:28.172583 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 15:59:28.172583 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 15:59:28.172583 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 15:59:28.172583 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:28.172583 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 15:59:28.172583 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:28.172583 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:28.172583 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 15:59:28.172583 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 15:59:28.172583 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 16 15:59:28.172583 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 16 15:59:28.172583 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 15:59:28.172583 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application next(self.gen) Jun 16 15:59:28.173865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 15:59:28.173865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application yield resource Jun 16 15:59:28.173865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 15:59:28.173865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application next(self.gen) Jun 16 15:59:28.173865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 15:59:28.173865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 15:59:28.173865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 15:59:28.173865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 15:59:28.173865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 15:59:28.173865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise self.value Jun 16 15:59:28.173865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 15:59:28.173865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 15:59:28.173865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 15:59:28.173865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application session.commit() Jun 16 15:59:28.173865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 15:59:28.173865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 15:59:28.173865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 15:59:28.175190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 15:59:28.175190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 15:59:28.175190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self.session.flush() Jun 16 15:59:28.175190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 15:59:28.175190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 15:59:28.175190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 15:59:28.175190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 15:59:28.175190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 15:59:28.175190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application compat.raise_( Jun 16 15:59:28.175190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 15:59:28.175190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception Jun 16 15:59:28.175190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 15:59:28.175190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 15:59:28.175190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 15:59:28.175190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 15:59:28.175190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 15:59:28.175190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 15:59:28.176113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 15:59:28.176113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 15:59:28.176113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 16 15:59:28.176113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 16 15:59:28.176113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 15:59:28.176113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 15:59:28.176113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 15:59:28.176113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 15:59:28.176113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 15:59:28.176113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 15:59:28.176113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 15:59:28.176113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 15:59:28.176113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 15:59:28.176113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 15:59:28.176113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 15:59:28.176113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception Jun 16 15:59:28.177157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 15:59:28.177157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 15:59:28.177157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 15:59:28.177157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 15:59:28.177157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 15:59:28.177157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 15:59:28.177157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 15:59:28.177157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application conn.query(q) Jun 16 15:59:28.177157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 15:59:28.177157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 15:59:28.177157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 15:59:28.177157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application result.read() Jun 16 15:59:28.177157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 15:59:28.177157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 15:59:28.177157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 15:59:28.177157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 15:59:28.178092 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 15:59:28.178092 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 15:59:28.178092 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 15:59:28.178092 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 15:59:28.178092 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 15:59:28.178092 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 15:59:28.178092 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [parameters: {'user_id': 'fd8485e411634625893ad36ecfc25a61', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 15:59:28.178092 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 15:59:28.178092 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:28.178092 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:59:28.178092 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:28.178092 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:28.178092 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:28.178092 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:28.178092 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:28.179013 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:28.179013 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:28.179013 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:28.179013 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:28.179013 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:28.179013 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:28.179013 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:28.179013 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 16 15:59:28.179013 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 16 15:59:28.179013 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:28.179013 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:28.179013 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:28.179013 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:28.179013 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:28.179013 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:28.179013 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 16 15:59:28.179013 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 16 15:59:28.179965 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 16 15:59:28.179965 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 16 15:59:28.179965 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 15:59:28.179965 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 15:59:28.179965 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 15:59:28.179965 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:28.182211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 476/958] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jun 16 15:59:27 2022] POST /identity/v3/users => generated 160 bytes in 209 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:28.188202 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4795abb-80ac-458f-bf6e-5929de175e95 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:28.307233 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:28.330761 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4795abb-80ac-458f-bf6e-5929de175e95 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:28.368221 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1004fd6-cc1e-4a7b-83ed-7adcc4031cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:28.369046 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1004fd6-cc1e-4a7b-83ed-7adcc4031cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:28.369338 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1004fd6-cc1e-4a7b-83ed-7adcc4031cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:28.369616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1004fd6-cc1e-4a7b-83ed-7adcc4031cd6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:28.371076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb4162-da5e-4c99-ac28-429a7759251c None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:28.371365 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb4162-da5e-4c99-ac28-429a7759251c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:28.371653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb4162-da5e-4c99-ac28-429a7759251c None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:28.376822 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 477/959] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:28.378596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1004fd6-cc1e-4a7b-83ed-7adcc4031cd6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:28.380050 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1004fd6-cc1e-4a7b-83ed-7adcc4031cd6 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:28.386774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acfbd1c-124c-40a4-8384-52b4d8dc36fd None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:28.387112 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acfbd1c-124c-40a4-8384-52b4d8dc36fd None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:28.387340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acfbd1c-124c-40a4-8384-52b4d8dc36fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:28.400142 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-a1004fd6-cc1e-4a7b-83ed-7adcc4031cd6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 15:59:28.400142 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:28.400142 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:28.400142 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:28.400142 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:28.400142 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:28.400142 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:28.400142 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:28.400142 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:28.400142 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:28.400142 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:28.400142 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:28.400142 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:28.400142 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:28.400142 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:28.400142 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:28.401163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:28.401163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:28.401163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:28.401163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:28.401163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:28.401163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:28.401163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:28.401163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:28.401163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:28.401163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:28.401163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:28.401163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:28.401163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:28.401163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:28.401163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:28.401163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:28.401163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:28.401163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:28.402236 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:28.402236 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:28.402236 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:28.402236 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:28.402236 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:28.402236 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:28.402236 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:28.402236 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:28.402236 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 15:59:28.402236 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:28.405971 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 483/960] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:28.411071 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1004fd6-cc1e-4a7b-83ed-7adcc4031cd6 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:28.412614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-5acfbd1c-124c-40a4-8384-52b4d8dc36fd None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:28.414267 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5acfbd1c-124c-40a4-8384-52b4d8dc36fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:28.433836 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:28.504733 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 478/961] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:28.508445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41d220-49a8-4603-bc05-07190267336b None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:28.508948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41d220-49a8-4603-bc05-07190267336b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:28.509870 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41d220-49a8-4603-bc05-07190267336b None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:28.512756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 479/962] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:28.520199 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2b600-15b5-4253-8e15-ec99af44bc02 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:28.521439 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2b600-15b5-4253-8e15-ec99af44bc02 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:28.521900 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2b600-15b5-4253-8e15-ec99af44bc02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:28.546087 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1ad2b600-15b5-4253-8e15-ec99af44bc02 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:28.547399 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1ad2b600-15b5-4253-8e15-ec99af44bc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:28.550290 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1004fd6-cc1e-4a7b-83ed-7adcc4031cd6 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:28.589115 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43f064-de18-4689-9fd3-469b2ae48e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:28.589801 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43f064-de18-4689-9fd3-469b2ae48e4b None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:28.590115 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43f064-de18-4689-9fd3-469b2ae48e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:28.590391 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43f064-de18-4689-9fd3-469b2ae48e4b None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:28.591211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43f064-de18-4689-9fd3-469b2ae48e4b None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:28.592150 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43f064-de18-4689-9fd3-469b2ae48e4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:28.593828 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43f064-de18-4689-9fd3-469b2ae48e4b None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:28.642062 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 480/963] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:28.642062 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2b600-15b5-4253-8e15-ec99af44bc02 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:28.658164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 484/964] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 15:59:28 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:28.691707 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43f064-de18-4689-9fd3-469b2ae48e4b None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:28.695473 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:28.778350 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:28.784860 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2b600-15b5-4253-8e15-ec99af44bc02 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:28.830903 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1719d4-0b80-4a26-9006-4d15fd1734c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:28.831567 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43f064-de18-4689-9fd3-469b2ae48e4b None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:28.832702 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1719d4-0b80-4a26-9006-4d15fd1734c2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:28.833252 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1719d4-0b80-4a26-9006-4d15fd1734c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:28.833749 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1719d4-0b80-4a26-9006-4d15fd1734c2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:28.835090 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1719d4-0b80-4a26-9006-4d15fd1734c2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:28.836754 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1719d4-0b80-4a26-9006-4d15fd1734c2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:28.851553 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-1e1719d4-0b80-4a26-9006-4d15fd1734c2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 15:59:28.851553 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:28.851553 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 15:59:28.851553 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 15:59:28.851553 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 15:59:28.851553 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 15:59:28.851553 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 15:59:28.851553 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:28.851553 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 15:59:28.851553 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:28.851553 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:59:28.851553 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:28.851553 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:28.851553 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:28.851553 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:28.851553 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:28.851553 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:28.852660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:28.852660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:28.852660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:28.852660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:28.852660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:28.852660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:28.852660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 15:59:28.852660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 15:59:28.852660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 15:59:28.852660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 15:59:28.852660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:28.852660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:28.852660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:28.852660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:28.852660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:28.852660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:28.853527 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:28.853527 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:28.853527 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:28.853527 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:28.853527 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:28.853527 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:28.853527 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:28.853527 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:28.853527 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:28.853527 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:28.853527 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 15:59:28.853527 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:28.853527 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 15:59:28.853527 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:28.859099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 481/965] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:28.860127 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1719d4-0b80-4a26-9006-4d15fd1734c2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:28.872374 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649e593-518c-418d-9f31-43f919935ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:28.873637 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649e593-518c-418d-9f31-43f919935ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:28.873637 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649e593-518c-418d-9f31-43f919935ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:28.873810 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649e593-518c-418d-9f31-43f919935ca8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:28.887455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5649e593-518c-418d-9f31-43f919935ca8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:28.896211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-5649e593-518c-418d-9f31-43f919935ca8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 15:59:28.896211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:28.896211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:28.896211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:28.896211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:28.896211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:28.896211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:28.896211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:28.896211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:28.896211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:28.896211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:28.896211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:28.896211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:28.896211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:28.896211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:28.896211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:28.898365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:28.898365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:28.898365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:28.898365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:28.898365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:28.898365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:28.898365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:28.898365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:28.898365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:28.898365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:28.898365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:28.898365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:28.898365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:28.898365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:28.898365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:28.898365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:28.900329 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:28.900329 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:28.900329 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:28.900329 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:28.900329 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:28.900329 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:28.900329 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:28.900329 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:28.900329 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:28.900329 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:28.900329 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 15:59:28.900329 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:28.902919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 485/966] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jun 16 15:59:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:28.904504 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649e593-518c-418d-9f31-43f919935ca8 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:28.996091 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:29.005617 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1719d4-0b80-4a26-9006-4d15fd1734c2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:29.045370 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d9530-b329-436d-973a-78d248843cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:29.046096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d9530-b329-436d-973a-78d248843cfa None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:29.046458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d9530-b329-436d-973a-78d248843cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:29.046744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d9530-b329-436d-973a-78d248843cfa None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:29.047473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d9530-b329-436d-973a-78d248843cfa None admin] RBAC: Adding query filter params (name=compute) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:29.047782 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d9530-b329-436d-973a-78d248843cfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:29.049180 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d9530-b329-436d-973a-78d248843cfa None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:29.053032 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649e593-518c-418d-9f31-43f919935ca8 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:29.054560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 482/967] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jun 16 15:59:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:29.060969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d9530-b329-436d-973a-78d248843cfa None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:29.074184 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:29.102177 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f0a55-5ba9-49df-9f50-ddfcefad1ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:29.103271 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f0a55-5ba9-49df-9f50-ddfcefad1ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:29.103685 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f0a55-5ba9-49df-9f50-ddfcefad1ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:29.104156 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f0a55-5ba9-49df-9f50-ddfcefad1ee4 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:29.105502 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f0a55-5ba9-49df-9f50-ddfcefad1ee4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:29.105893 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f0a55-5ba9-49df-9f50-ddfcefad1ee4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:29.108202 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f0a55-5ba9-49df-9f50-ddfcefad1ee4 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:29.117418 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 486/968] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jun 16 15:59:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:29.123569 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f0a55-5ba9-49df-9f50-ddfcefad1ee4 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:29.194282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d9530-b329-436d-973a-78d248843cfa None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:29.232746 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01a9f3-dbdc-4c5c-947d-0a4f32549c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:29.233430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01a9f3-dbdc-4c5c-947d-0a4f32549c70 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:29.233732 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01a9f3-dbdc-4c5c-947d-0a4f32549c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:29.234015 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01a9f3-dbdc-4c5c-947d-0a4f32549c70 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:29.234739 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01a9f3-dbdc-4c5c-947d-0a4f32549c70 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:29.235058 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01a9f3-dbdc-4c5c-947d-0a4f32549c70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:29.236450 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01a9f3-dbdc-4c5c-947d-0a4f32549c70 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:29.241420 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 483/969] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jun 16 15:59:29 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:29.242766 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f491b64-9ab3-469a-a7b9-4cec536a09cd None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:29.243056 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f491b64-9ab3-469a-a7b9-4cec536a09cd None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:29.243334 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f491b64-9ab3-469a-a7b9-4cec536a09cd None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:29.244627 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 484/970] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:29.251242 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd15f3b-f965-452f-b5fd-9c6c94b288af None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:29.251579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd15f3b-f965-452f-b5fd-9c6c94b288af None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:29.251843 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd15f3b-f965-452f-b5fd-9c6c94b288af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:29.255412 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f0a55-5ba9-49df-9f50-ddfcefad1ee4 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:29.271767 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ebd15f3b-f965-452f-b5fd-9c6c94b288af None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:29.273068 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ebd15f3b-f965-452f-b5fd-9c6c94b288af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:29.288757 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:29.306549 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9201c78-4b37-4221-bcc1-74dfc51865ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:29.307276 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9201c78-4b37-4221-bcc1-74dfc51865ac None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:29.307594 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9201c78-4b37-4221-bcc1-74dfc51865ac None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:29.307998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9201c78-4b37-4221-bcc1-74dfc51865ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:29.326229 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9201c78-4b37-4221-bcc1-74dfc51865ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:29.327889 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9201c78-4b37-4221-bcc1-74dfc51865ac None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:29.351012 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-e9201c78-4b37-4221-bcc1-74dfc51865ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:29.351012 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:29.351012 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:29.351012 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:29.351012 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:29.351012 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:29.351012 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:29.351012 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:29.351012 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:29.351012 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:29.351012 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:29.351012 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:29.351012 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:29.351012 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:29.351012 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:29.351998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:29.351998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:29.351998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:29.351998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:29.351998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:29.351998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:29.351998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:29.351998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:29.351998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:29.351998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:29.351998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:29.351998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:29.351998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:29.351998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:29.351998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:29.351998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:29.351998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:29.353149 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:29.353149 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:29.353149 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:29.353149 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:29.353149 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 487/971] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:29.359129 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9201c78-4b37-4221-bcc1-74dfc51865ac None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:29.370466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 485/972] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 15:59:29.396932 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd15f3b-f965-452f-b5fd-9c6c94b288af None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:29.399218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:29.498340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9201c78-4b37-4221-bcc1-74dfc51865ac None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:29.527946 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd15f3b-f965-452f-b5fd-9c6c94b288af None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:29.543990 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f992f9-ea88-4cbe-8ced-f744f1b43c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:29.545351 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f992f9-ea88-4cbe-8ced-f744f1b43c73 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:29.546027 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f992f9-ea88-4cbe-8ced-f744f1b43c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:29.546027 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f992f9-ea88-4cbe-8ced-f744f1b43c73 None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:29.546632 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f992f9-ea88-4cbe-8ced-f744f1b43c73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:29.546981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f992f9-ea88-4cbe-8ced-f744f1b43c73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:29.548457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f992f9-ea88-4cbe-8ced-f744f1b43c73 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:29.563564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 488/973] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:29.569219 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f992f9-ea88-4cbe-8ced-f744f1b43c73 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:29.573052 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7babc4-3033-4c30-9db0-509f19192646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:29.573905 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7babc4-3033-4c30-9db0-509f19192646 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:29.574216 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7babc4-3033-4c30-9db0-509f19192646 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:29.574493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7babc4-3033-4c30-9db0-509f19192646 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:29.575171 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7babc4-3033-4c30-9db0-509f19192646 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:29.576652 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7babc4-3033-4c30-9db0-509f19192646 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:29.590597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-4f7babc4-3033-4c30-9db0-509f19192646 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 15:59:29.590597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:29.590597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 15:59:29.590597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 15:59:29.590597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 15:59:29.590597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 15:59:29.590597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 15:59:29.590597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:29.590597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 15:59:29.590597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:29.590597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:59:29.590597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:29.590597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:29.590597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:29.590597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:29.590597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:29.590597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:29.593094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:29.593094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:29.593094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:29.593094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:29.593094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:29.593094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:29.593094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 15:59:29.593094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 15:59:29.593094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 15:59:29.593094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 15:59:29.593094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:29.593094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:29.593094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:29.593094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:29.593094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:29.593094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:29.595157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:29.595157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:29.595157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:29.595157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:29.595157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:29.595157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:29.595157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:29.595157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:29.595157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:29.595157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:29.595157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 15:59:29.595157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:29.595157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 15:59:29.595157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:29.595157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 486/974] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jun 16 15:59:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:29.599530 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7babc4-3033-4c30-9db0-509f19192646 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:29.622398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:29.679125 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:29.705005 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f992f9-ea88-4cbe-8ced-f744f1b43c73 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:29.724056 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7babc4-3033-4c30-9db0-509f19192646 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:29.746871 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a63100-2708-4fe0-b7e2-4a4a9617f30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:29.748114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a63100-2708-4fe0-b7e2-4a4a9617f30e None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:29.748506 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a63100-2708-4fe0-b7e2-4a4a9617f30e None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:29.748874 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a63100-2708-4fe0-b7e2-4a4a9617f30e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:29.757215 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a63100-2708-4fe0-b7e2-4a4a9617f30e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:29.758883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a63100-2708-4fe0-b7e2-4a4a9617f30e None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:29.769495 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe772dd-b20d-4abc-a082-498d6d6cc633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:29.770246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe772dd-b20d-4abc-a082-498d6d6cc633 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:29.770538 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe772dd-b20d-4abc-a082-498d6d6cc633 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:29.770820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe772dd-b20d-4abc-a082-498d6d6cc633 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:29.771779 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe772dd-b20d-4abc-a082-498d6d6cc633 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:29.772090 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe772dd-b20d-4abc-a082-498d6d6cc633 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:29.773524 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe772dd-b20d-4abc-a082-498d6d6cc633 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:29.777879 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-d1a63100-2708-4fe0-b7e2-4a4a9617f30e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 15:59:29.777879 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:29.777879 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:29.777879 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:29.777879 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:29.777879 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:29.777879 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:29.777879 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:29.777879 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:29.777879 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:29.777879 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:29.777879 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:29.777879 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:29.777879 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:29.777879 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:29.777879 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:29.780034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:29.780034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:29.780034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:29.780034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:29.780034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:29.780034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:29.780034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:29.780034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:29.780034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:29.780034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:29.780034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:29.780034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:29.780034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:29.780034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:29.780034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:29.780034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:29.780034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:29.780034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:29.781136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:29.781136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:29.781136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:29.781136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:29.781136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:29.781136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:29.781136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:29.781136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:29.781136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 15:59:29.781136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:29.788065 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 489/975] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jun 16 15:59:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:29.788790 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 487/976] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jun 16 15:59:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:29.789774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a63100-2708-4fe0-b7e2-4a4a9617f30e None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:29.793368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe772dd-b20d-4abc-a082-498d6d6cc633 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:29.920852 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:29.928245 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a63100-2708-4fe0-b7e2-4a4a9617f30e None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:29.938601 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe772dd-b20d-4abc-a082-498d6d6cc633 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:29.969536 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f31b7-6c47-4bf9-a200-431a97e4c6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:29.970247 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f31b7-6c47-4bf9-a200-431a97e4c6ef None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:29.970531 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f31b7-6c47-4bf9-a200-431a97e4c6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:29.970802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f31b7-6c47-4bf9-a200-431a97e4c6ef None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:29.970905 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:29.971652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f31b7-6c47-4bf9-a200-431a97e4c6ef None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:29.972008 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f31b7-6c47-4bf9-a200-431a97e4c6ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:29.973809 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f31b7-6c47-4bf9-a200-431a97e4c6ef None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:29.979672 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee5c8e-51d3-4358-b140-f9704ad08c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:29.980955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee5c8e-51d3-4358-b140-f9704ad08c07 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:29.981442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee5c8e-51d3-4358-b140-f9704ad08c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:29.981989 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee5c8e-51d3-4358-b140-f9704ad08c07 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:29.982918 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee5c8e-51d3-4358-b140-f9704ad08c07 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:29.983597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee5c8e-51d3-4358-b140-f9704ad08c07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:29.986213 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee5c8e-51d3-4358-b140-f9704ad08c07 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:29.998208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 490/977] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jun 16 15:59:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:29.998802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 488/978] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jun 16 15:59:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:29.998896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f31b7-6c47-4bf9-a200-431a97e4c6ef None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:29.999492 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee5c8e-51d3-4358-b140-f9704ad08c07 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:30.115677 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f31b7-6c47-4bf9-a200-431a97e4c6ef None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:30.121570 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee5c8e-51d3-4358-b140-f9704ad08c07 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:30.156433 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73795cb6-4fa1-4dc1-86c8-e6bc3cc99c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:30.157017 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73795cb6-4fa1-4dc1-86c8-e6bc3cc99c84 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:30.157332 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73795cb6-4fa1-4dc1-86c8-e6bc3cc99c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:30.157566 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73795cb6-4fa1-4dc1-86c8-e6bc3cc99c84 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:30.158396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73795cb6-4fa1-4dc1-86c8-e6bc3cc99c84 None admin] RBAC: Adding query filter params (interface=admin, service_id=079a7542c1374120ba40e2b0d3f58513, region_id=RegionOne) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:30.158709 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73795cb6-4fa1-4dc1-86c8-e6bc3cc99c84 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:30.159824 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95558d25-e123-449e-8ba6-333b20449987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:30.160094 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73795cb6-4fa1-4dc1-86c8-e6bc3cc99c84 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:30.160506 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95558d25-e123-449e-8ba6-333b20449987 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:30.160787 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95558d25-e123-449e-8ba6-333b20449987 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:30.161057 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95558d25-e123-449e-8ba6-333b20449987 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:30.167102 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 491/979] 10.222.0.17 () {56 vars in 1249 bytes} [Thu Jun 16 15:59:29 2022] GET /identity/v3/endpoints?service_id=079a7542c1374120ba40e2b0d3f58513&interface=admin®ion_id=RegionOne => generated 193 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:30.172663 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73795cb6-4fa1-4dc1-86c8-e6bc3cc99c84 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:30.195910 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:30.260914 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:30.271302 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95558d25-e123-449e-8ba6-333b20449987 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:30.272976 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95558d25-e123-449e-8ba6-333b20449987 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:30.291277 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73795cb6-4fa1-4dc1-86c8-e6bc3cc99c84 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:30.333953 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b56354-0ebd-4925-b945-5f59c59a8ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:30.334809 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b56354-0ebd-4925-b945-5f59c59a8ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:30.335233 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b56354-0ebd-4925-b945-5f59c59a8ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:30.335639 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b56354-0ebd-4925-b945-5f59c59a8ac5 None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:30.337170 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b56354-0ebd-4925-b945-5f59c59a8ac5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:30.338930 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b56354-0ebd-4925-b945-5f59c59a8ac5 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:30.361965 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 492/980] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jun 16 15:59:30 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 15:59:30.401112 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 489/981] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:29 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 15:59:30.407540 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b56354-0ebd-4925-b945-5f59c59a8ac5 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:30.516255 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:30.545779 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b56354-0ebd-4925-b945-5f59c59a8ac5 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:30.591032 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893b084-04a8-4875-b25b-ccb4f6e8c2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:30.592397 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893b084-04a8-4875-b25b-ccb4f6e8c2ec None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:30.592680 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893b084-04a8-4875-b25b-ccb4f6e8c2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:30.592974 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893b084-04a8-4875-b25b-ccb4f6e8c2ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:30.604839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b893b084-04a8-4875-b25b-ccb4f6e8c2ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:30.606390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b893b084-04a8-4875-b25b-ccb4f6e8c2ec None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:30.630847 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-b893b084-04a8-4875-b25b-ccb4f6e8c2ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:30.630847 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:30.630847 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:30.630847 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:30.630847 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:30.630847 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:30.630847 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:30.630847 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:30.630847 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:30.630847 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:30.630847 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:30.630847 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:30.630847 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:30.630847 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:30.630847 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:30.632732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:30.632732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:30.632732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:30.632732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:30.632732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:30.632732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:30.632732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:30.632732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:30.632732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:30.632732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:30.632732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:30.632732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:30.632732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:30.632732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:30.632732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:30.632732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:30.632732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:30.635416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:30.635416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:30.635416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:30.635416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:30.635416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 493/982] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:30.639355 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95558d25-e123-449e-8ba6-333b20449987 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:30.809020 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:30.819936 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95558d25-e123-449e-8ba6-333b20449987 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:30.886686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5e1d2-d27a-4891-aa6f-fca5d20ec88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:30.887666 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5e1d2-d27a-4891-aa6f-fca5d20ec88d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:30.888856 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5e1d2-d27a-4891-aa6f-fca5d20ec88d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:30.889363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5e1d2-d27a-4891-aa6f-fca5d20ec88d None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:30.890750 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5e1d2-d27a-4891-aa6f-fca5d20ec88d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:30.891315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5e1d2-d27a-4891-aa6f-fca5d20ec88d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:30.893669 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5e1d2-d27a-4891-aa6f-fca5d20ec88d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:30.921010 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 490/983] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:30.929679 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893b084-04a8-4875-b25b-ccb4f6e8c2ec None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:31.087468 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893b084-04a8-4875-b25b-ccb4f6e8c2ec None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:31.130880 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f50245-7e14-4cfb-ab85-dfb767f8a0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:31.131730 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f50245-7e14-4cfb-ab85-dfb767f8a0e1 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:31.132014 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f50245-7e14-4cfb-ab85-dfb767f8a0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:31.132507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f50245-7e14-4cfb-ab85-dfb767f8a0e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:31.134985 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:31.143303 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f50245-7e14-4cfb-ab85-dfb767f8a0e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:31.144800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f50245-7e14-4cfb-ab85-dfb767f8a0e1 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:31.161621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-49f50245-7e14-4cfb-ab85-dfb767f8a0e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:31.161621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:31.161621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:31.161621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:31.161621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:31.161621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:31.161621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:31.161621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:31.161621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:31.161621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:31.161621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:31.161621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:31.161621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:31.161621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:31.161621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:31.163226 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:31.163226 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:31.163226 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:31.163226 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:31.163226 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:31.163226 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:31.163226 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:31.163226 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:31.163226 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:31.163226 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:31.163226 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:31.163226 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:31.163226 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:31.163226 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:31.163226 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:31.163226 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:31.163226 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:31.164315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:31.164315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:31.164315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:31.164315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:31.164315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:31.164315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:31.164315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:31.164315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:31.165695 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 494/984] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:31.169424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5e1d2-d27a-4891-aa6f-fca5d20ec88d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:31.343250 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5e1d2-d27a-4891-aa6f-fca5d20ec88d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:31.382961 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c7e7c-e4b7-4a54-b37e-d66aa40fc461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:31.384301 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c7e7c-e4b7-4a54-b37e-d66aa40fc461 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:31.384301 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c7e7c-e4b7-4a54-b37e-d66aa40fc461 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:31.384301 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c7e7c-e4b7-4a54-b37e-d66aa40fc461 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:31.385418 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c7e7c-e4b7-4a54-b37e-d66aa40fc461 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:31.385622 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c7e7c-e4b7-4a54-b37e-d66aa40fc461 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:31.387188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c7e7c-e4b7-4a54-b37e-d66aa40fc461 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:31.417873 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 491/985] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:31.418971 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f50245-7e14-4cfb-ab85-dfb767f8a0e1 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:31.494866 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4d7ee-d3fb-4a1d-b925-a2dc8c5b3600 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:31.495219 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4d7ee-d3fb-4a1d-b925-a2dc8c5b3600 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:31.495436 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4d7ee-d3fb-4a1d-b925-a2dc8c5b3600 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:31.496974 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 492/986] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:31.503203 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d07c05-126b-45c3-8c25-c25f9ac8ef6b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:31.503447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d07c05-126b-45c3-8c25-c25f9ac8ef6b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:31.503689 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d07c05-126b-45c3-8c25-c25f9ac8ef6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:31.522772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-33d07c05-126b-45c3-8c25-c25f9ac8ef6b None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:31.523986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-33d07c05-126b-45c3-8c25-c25f9ac8ef6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:31.536117 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:31.577324 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f50245-7e14-4cfb-ab85-dfb767f8a0e1 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:31.620968 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 493/987] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 15:59:31.636166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64125c-25f5-4cc9-9b27-4eb6c27df4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:31.636901 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64125c-25f5-4cc9-9b27-4eb6c27df4a9 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:31.637236 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64125c-25f5-4cc9-9b27-4eb6c27df4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:31.637511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64125c-25f5-4cc9-9b27-4eb6c27df4a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:31.639244 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c64125c-25f5-4cc9-9b27-4eb6c27df4a9 None admin] RBAC: Adding query filter params (role.id=758069bda1404a3589771a8f43e95187, scope.project.id=b1177a85a0de4276898daf4a24ff069f, user.id=2c2e0e19880f4b048faf034dd38c7365) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:31.639762 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c64125c-25f5-4cc9-9b27-4eb6c27df4a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:31.642597 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c64125c-25f5-4cc9-9b27-4eb6c27df4a9 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:31.645246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d07c05-126b-45c3-8c25-c25f9ac8ef6b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:31.653692 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 495/988] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:59:31 2022] GET /identity/v3/role_assignments?user.id=2c2e0e19880f4b048faf034dd38c7365&scope.project.id=b1177a85a0de4276898daf4a24ff069f&role.id=758069bda1404a3589771a8f43e95187 => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:31.789027 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d07c05-126b-45c3-8c25-c25f9ac8ef6b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:31.837678 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0535a34-b78d-4e19-be54-b25f060223fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:31.838419 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0535a34-b78d-4e19-be54-b25f060223fc None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:31.838699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0535a34-b78d-4e19-be54-b25f060223fc None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:31.838978 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0535a34-b78d-4e19-be54-b25f060223fc None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:31.840948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0535a34-b78d-4e19-be54-b25f060223fc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:31.841141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0535a34-b78d-4e19-be54-b25f060223fc None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:31.860463 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 494/989] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jun 16 15:59:31 2022] POST /identity/v3/services => generated 242 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:59:32.687102 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1abc08e-a446-4de7-8dd6-5ee79893cebe None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:32.687664 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1abc08e-a446-4de7-8dd6-5ee79893cebe None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:32.687664 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1abc08e-a446-4de7-8dd6-5ee79893cebe None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:32.689293 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 496/990] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:32.694717 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef1db8-4b86-436e-9805-fca7c647c523 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:32.695158 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef1db8-4b86-436e-9805-fca7c647c523 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:32.695261 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef1db8-4b86-436e-9805-fca7c647c523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:32.703614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:32.715850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-bcef1db8-4b86-436e-9805-fca7c647c523 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:32.717701 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bcef1db8-4b86-436e-9805-fca7c647c523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:32.803106 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 495/991] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:32 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:32.828406 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1abc08e-a446-4de7-8dd6-5ee79893cebe None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:32.894267 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:32.942338 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1abc08e-a446-4de7-8dd6-5ee79893cebe None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:32.977321 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a98cf7-38b7-4f90-adf6-560c40bc7e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:32.978155 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a98cf7-38b7-4f90-adf6-560c40bc7e41 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:32.978443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a98cf7-38b7-4f90-adf6-560c40bc7e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:32.978720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a98cf7-38b7-4f90-adf6-560c40bc7e41 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:32.979380 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a98cf7-38b7-4f90-adf6-560c40bc7e41 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:32.980769 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a98cf7-38b7-4f90-adf6-560c40bc7e41 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:32.992882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-47a98cf7-38b7-4f90-adf6-560c40bc7e41 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 15:59:32.992882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:32.992882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 15:59:32.992882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 15:59:32.992882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 15:59:32.992882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 15:59:32.992882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 15:59:32.992882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:32.992882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 15:59:32.992882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:32.992882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:59:32.992882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:32.992882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:32.992882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:32.992882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:32.992882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:32.992882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:32.993885 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:32.993885 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:32.993885 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:32.993885 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:32.993885 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:32.993885 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:32.993885 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 15:59:32.993885 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 15:59:32.993885 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 15:59:32.993885 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 15:59:32.993885 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:32.993885 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:32.993885 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:32.993885 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:32.993885 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:32.993885 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:32.994781 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:32.994781 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:32.994781 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:32.994781 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:32.994781 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:32.994781 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:32.994781 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:32.994781 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:32.994781 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:32.994781 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:32.994781 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 15:59:32.994781 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:32.994781 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 15:59:32.994781 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:32.994781 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 497/992] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jun 16 15:59:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:33.000560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef1db8-4b86-436e-9805-fca7c647c523 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:33.111973 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef1db8-4b86-436e-9805-fca7c647c523 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:33.143281 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:33.150150 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c2489-9bdb-4568-bfe3-b123e1e97060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:33.150819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c2489-9bdb-4568-bfe3-b123e1e97060 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:33.151098 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c2489-9bdb-4568-bfe3-b123e1e97060 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:33.151370 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c2489-9bdb-4568-bfe3-b123e1e97060 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:33.152071 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c2489-9bdb-4568-bfe3-b123e1e97060 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:33.152378 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c2489-9bdb-4568-bfe3-b123e1e97060 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:33.154761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c2489-9bdb-4568-bfe3-b123e1e97060 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:33.160369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 496/993] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jun 16 15:59:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:33.165914 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a98cf7-38b7-4f90-adf6-560c40bc7e41 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:33.271380 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a98cf7-38b7-4f90-adf6-560c40bc7e41 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:33.295036 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:33.307533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d82239-0379-4dec-a543-3fd58cf28697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:33.308192 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d82239-0379-4dec-a543-3fd58cf28697 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:33.308473 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d82239-0379-4dec-a543-3fd58cf28697 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:33.308769 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d82239-0379-4dec-a543-3fd58cf28697 None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:33.309470 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d82239-0379-4dec-a543-3fd58cf28697 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:33.309800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d82239-0379-4dec-a543-3fd58cf28697 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:33.312482 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d82239-0379-4dec-a543-3fd58cf28697 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:33.317639 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 498/994] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jun 16 15:59:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:33.323180 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c2489-9bdb-4568-bfe3-b123e1e97060 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:33.427582 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c2489-9bdb-4568-bfe3-b123e1e97060 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:33.464371 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139fa8f1-a50d-48ed-85d2-f43badacc8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:33.465041 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fa8f1-a50d-48ed-85d2-f43badacc8a9 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:33.465327 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fa8f1-a50d-48ed-85d2-f43badacc8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:33.465606 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fa8f1-a50d-48ed-85d2-f43badacc8a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:33.467036 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139fa8f1-a50d-48ed-85d2-f43badacc8a9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:33.468485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139fa8f1-a50d-48ed-85d2-f43badacc8a9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:33.489956 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 497/995] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jun 16 15:59:33 2022] POST /identity/v3/endpoints => generated 325 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:59:33.918967 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd449c-626f-4f43-8a4b-663fe1f0bd3b None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:33.919569 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd449c-626f-4f43-8a4b-663fe1f0bd3b None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:33.919569 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd449c-626f-4f43-8a4b-663fe1f0bd3b None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:33.920944 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 499/996] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:33.926418 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28690aa3-e7cb-486b-9858-75644c70d7cb None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:33.926850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28690aa3-e7cb-486b-9858-75644c70d7cb None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:33.927061 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28690aa3-e7cb-486b-9858-75644c70d7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:33.944918 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-28690aa3-e7cb-486b-9858-75644c70d7cb None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:33.946660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-28690aa3-e7cb-486b-9858-75644c70d7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:34.001357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:34.037709 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 498/997] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:33 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:34.060811 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd449c-626f-4f43-8a4b-663fe1f0bd3b None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:34.079457 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f07410-cdb8-4fa0-8931-13d34e12bcc5 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:34.080379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f07410-cdb8-4fa0-8931-13d34e12bcc5 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:34.080661 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f07410-cdb8-4fa0-8931-13d34e12bcc5 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:34.082531 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 499/998] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:59:34.087633 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8aca1d-0bcd-4add-bd7a-9dccb6656f59 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:34.087954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8aca1d-0bcd-4add-bd7a-9dccb6656f59 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:34.088194 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8aca1d-0bcd-4add-bd7a-9dccb6656f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:34.105411 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ae8aca1d-0bcd-4add-bd7a-9dccb6656f59 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:34.107170 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ae8aca1d-0bcd-4add-bd7a-9dccb6656f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:34.178888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 500/999] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:34 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:34.182685 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd449c-626f-4f43-8a4b-663fe1f0bd3b None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:34.202449 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8aca1d-0bcd-4add-bd7a-9dccb6656f59 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:34.219718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6ccf6-84ad-4d5d-bea8-78b5de30400a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:34.220519 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6ccf6-84ad-4d5d-bea8-78b5de30400a None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:34.220801 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6ccf6-84ad-4d5d-bea8-78b5de30400a None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:34.221078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6ccf6-84ad-4d5d-bea8-78b5de30400a None admin] PATH_INFO: `/v3/services/image` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:34.221727 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b6ccf6-84ad-4d5d-bea8-78b5de30400a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:34.223161 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b6ccf6-84ad-4d5d-bea8-78b5de30400a None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:34.235694 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-f3b6ccf6-84ad-4d5d-bea8-78b5de30400a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 15:59:34.235694 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:34.235694 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 15:59:34.235694 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 15:59:34.235694 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 15:59:34.235694 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 15:59:34.235694 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 15:59:34.235694 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:34.235694 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 15:59:34.235694 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:34.235694 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:59:34.235694 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:34.235694 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:34.235694 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:34.235694 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:34.235694 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:34.235694 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:34.236739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:34.236739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:34.236739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:34.236739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:34.236739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:34.236739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:34.236739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 15:59:34.236739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 15:59:34.236739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 15:59:34.236739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 15:59:34.236739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:34.236739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:34.236739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:34.236739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:34.236739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:34.236739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:34.237596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:34.237596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:34.237596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:34.237596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:34.237596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:34.237596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:34.237596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:34.237596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:34.237596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:34.237596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:34.237596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 15:59:34.237596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:34.237596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 15:59:34.237596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:34.237596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 500/1000] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jun 16 15:59:34 2022] GET /identity/v3/services/image => generated 86 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:34.242628 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6ccf6-84ad-4d5d-bea8-78b5de30400a None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:34.299607 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:34.323397 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8aca1d-0bcd-4add-bd7a-9dccb6656f59 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:34.337196 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:34.355718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6ccf6-84ad-4d5d-bea8-78b5de30400a None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:34.365484 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50217c5-f16e-4488-9826-288c3eb4025b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:34.366329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50217c5-f16e-4488-9826-288c3eb4025b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:34.366609 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50217c5-f16e-4488-9826-288c3eb4025b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:34.366883 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50217c5-f16e-4488-9826-288c3eb4025b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:34.367542 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50217c5-f16e-4488-9826-288c3eb4025b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:34.368912 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50217c5-f16e-4488-9826-288c3eb4025b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:34.381227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-a50217c5-f16e-4488-9826-288c3eb4025b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 15:59:34.381227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:34.381227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 15:59:34.381227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 15:59:34.381227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 15:59:34.381227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 15:59:34.381227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 15:59:34.381227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:34.381227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 15:59:34.381227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:34.381227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:59:34.381227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:34.381227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:34.381227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:34.381227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:34.381227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:34.381227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:34.382455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:34.382455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:34.382455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:34.382455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:34.382455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:34.382455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:34.382455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 15:59:34.382455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 15:59:34.382455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 15:59:34.382455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 15:59:34.382455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:34.382455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:34.382455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:34.382455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:34.382455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:34.382455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:34.383430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:34.383430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:34.383430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:34.383430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:34.383430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:34.383430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:34.383430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:34.383430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:34.383430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:34.383430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:34.383430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 15:59:34.383430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:34.383430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 15:59:34.383430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:34.383430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 501/1001] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:34.389447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50217c5-f16e-4488-9826-288c3eb4025b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:34.399999 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ef021-5868-45d3-8aff-844b933844f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:34.401339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ef021-5868-45d3-8aff-844b933844f6 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:34.401770 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ef021-5868-45d3-8aff-844b933844f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:34.402240 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ef021-5868-45d3-8aff-844b933844f6 None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:34.403081 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ef021-5868-45d3-8aff-844b933844f6 None admin] RBAC: Adding query filter params (name=image) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:34.403664 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ef021-5868-45d3-8aff-844b933844f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:34.406322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ef021-5868-45d3-8aff-844b933844f6 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:34.411800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 501/1002] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:34.416943 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ef021-5868-45d3-8aff-844b933844f6 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:34.499175 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50217c5-f16e-4488-9826-288c3eb4025b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:34.532343 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ef021-5868-45d3-8aff-844b933844f6 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:34.540343 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05fc3eb-097a-4614-870a-6887779f71e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:34.541011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05fc3eb-097a-4614-870a-6887779f71e7 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:34.541292 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05fc3eb-097a-4614-870a-6887779f71e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:34.541558 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05fc3eb-097a-4614-870a-6887779f71e7 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:34.542288 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05fc3eb-097a-4614-870a-6887779f71e7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:34.542591 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05fc3eb-097a-4614-870a-6887779f71e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:34.543962 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05fc3eb-097a-4614-870a-6887779f71e7 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:34.547559 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:34.554209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 502/1003] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jun 16 15:59:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:34.555358 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05fc3eb-097a-4614-870a-6887779f71e7 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:34.576685 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f7b17-ba1a-49c3-9ec4-c602441360f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:34.577439 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f7b17-ba1a-49c3-9ec4-c602441360f1 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:34.577723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f7b17-ba1a-49c3-9ec4-c602441360f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:34.578021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f7b17-ba1a-49c3-9ec4-c602441360f1 None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:34.578731 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f7b17-ba1a-49c3-9ec4-c602441360f1 None admin] RBAC: Adding query filter params (type=image) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:34.579041 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f7b17-ba1a-49c3-9ec4-c602441360f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:34.580410 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f7b17-ba1a-49c3-9ec4-c602441360f1 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:34.585204 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 502/1004] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:34 2022] GET /identity/v3/services?type=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:34.612096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:34.684583 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05fc3eb-097a-4614-870a-6887779f71e7 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:34.724835 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e009a-9ab5-4d17-9a0d-4d3ffa4bc56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:34.725529 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e009a-9ab5-4d17-9a0d-4d3ffa4bc56b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:34.725814 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e009a-9ab5-4d17-9a0d-4d3ffa4bc56b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:34.726119 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e009a-9ab5-4d17-9a0d-4d3ffa4bc56b None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:34.726838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e009a-9ab5-4d17-9a0d-4d3ffa4bc56b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:34.727145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e009a-9ab5-4d17-9a0d-4d3ffa4bc56b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:34.728537 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e009a-9ab5-4d17-9a0d-4d3ffa4bc56b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:34.735067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 503/1005] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jun 16 15:59:34 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:34.740901 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f7b17-ba1a-49c3-9ec4-c602441360f1 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:34.866554 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f7b17-ba1a-49c3-9ec4-c602441360f1 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:34.902316 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c8fef8-a047-4883-b451-3c1fcae2c449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:34.902971 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8fef8-a047-4883-b451-3c1fcae2c449 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:34.903251 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8fef8-a047-4883-b451-3c1fcae2c449 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:34.904149 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8fef8-a047-4883-b451-3c1fcae2c449 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:34.905381 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c8fef8-a047-4883-b451-3c1fcae2c449 None admin] RBAC: Adding query filter params (interface=public, service_id=63a21faab0b14cd0bf095cf4d0353327, region_id=RegionOne) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:34.905693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c8fef8-a047-4883-b451-3c1fcae2c449 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:34.907094 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c8fef8-a047-4883-b451-3c1fcae2c449 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:34.912659 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 503/1006] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jun 16 15:59:34 2022] GET /identity/v3/endpoints?service_id=63a21faab0b14cd0bf095cf4d0353327&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:34.917883 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e009a-9ab5-4d17-9a0d-4d3ffa4bc56b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:35.025951 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e009a-9ab5-4d17-9a0d-4d3ffa4bc56b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:35.027526 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:35.063403 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a73a81-b6e5-42bf-a48f-a5429ea592e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:35.064153 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a73a81-b6e5-42bf-a48f-a5429ea592e4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:35.064536 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a73a81-b6e5-42bf-a48f-a5429ea592e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:35.064783 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a73a81-b6e5-42bf-a48f-a5429ea592e4 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:35.065474 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a73a81-b6e5-42bf-a48f-a5429ea592e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:35.067043 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a73a81-b6e5-42bf-a48f-a5429ea592e4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:35.075064 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 504/1007] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jun 16 15:59:34 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:59:35.721424 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1185556-d4f7-4a02-a32a-b7eb996d6762 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:35.721968 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1185556-d4f7-4a02-a32a-b7eb996d6762 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:35.722158 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1185556-d4f7-4a02-a32a-b7eb996d6762 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:35.723484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 504/1008] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:35.729339 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce22df-f6be-4f18-b47c-13b0371dfc65 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:35.729635 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce22df-f6be-4f18-b47c-13b0371dfc65 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:35.729977 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce22df-f6be-4f18-b47c-13b0371dfc65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:35.748494 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1bce22df-f6be-4f18-b47c-13b0371dfc65 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:35.749708 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1bce22df-f6be-4f18-b47c-13b0371dfc65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:35.828705 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 505/1009] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:35 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:35.853203 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1185556-d4f7-4a02-a32a-b7eb996d6762 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:35.904289 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:35.966941 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1185556-d4f7-4a02-a32a-b7eb996d6762 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:36.000852 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9f430-1c65-4ef0-a5aa-6cc3a49d7004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:36.001619 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9f430-1c65-4ef0-a5aa-6cc3a49d7004 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:36.001909 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9f430-1c65-4ef0-a5aa-6cc3a49d7004 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:36.002314 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9f430-1c65-4ef0-a5aa-6cc3a49d7004 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:36.003505 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9f430-1c65-4ef0-a5aa-6cc3a49d7004 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:36.005002 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9f430-1c65-4ef0-a5aa-6cc3a49d7004 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:36.006946 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 505/1010] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:36.013418 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bce22df-f6be-4f18-b47c-13b0371dfc65 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:36.110625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:36.117010 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bce22df-f6be-4f18-b47c-13b0371dfc65 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:36.151967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b75c8-89e8-4947-bb47-03d642b78e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:36.152628 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b75c8-89e8-4947-bb47-03d642b78e40 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:36.152909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b75c8-89e8-4947-bb47-03d642b78e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:36.153182 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b75c8-89e8-4947-bb47-03d642b78e40 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:36.153891 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39b75c8-89e8-4947-bb47-03d642b78e40 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:36.155350 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39b75c8-89e8-4947-bb47-03d642b78e40 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:36.180753 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 506/1011] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jun 16 15:59:36 2022] POST /identity/v3/projects => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:59:36.826279 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb710a-56b1-477a-b923-e36ac4e7a5df None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:36.826792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb710a-56b1-477a-b923-e36ac4e7a5df None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:36.826901 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb710a-56b1-477a-b923-e36ac4e7a5df None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:36.828447 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 506/1012] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:36.833466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c6a1a-01f6-4ac8-b0aa-e77ae33801a1 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:36.833760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c6a1a-01f6-4ac8-b0aa-e77ae33801a1 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:36.834082 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c6a1a-01f6-4ac8-b0aa-e77ae33801a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:36.851105 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-940c6a1a-01f6-4ac8-b0aa-e77ae33801a1 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:36.852287 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-940c6a1a-01f6-4ac8-b0aa-e77ae33801a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:36.924900 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 507/1013] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:36 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:36.953502 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fb710a-56b1-477a-b923-e36ac4e7a5df None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:37.068857 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fb710a-56b1-477a-b923-e36ac4e7a5df None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:37.080363 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:37.105807 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49851c-9f78-464c-9eb1-12a9b1c04c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:37.106516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49851c-9f78-464c-9eb1-12a9b1c04c9e None admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:37.106990 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49851c-9f78-464c-9eb1-12a9b1c04c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:37.107400 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49851c-9f78-464c-9eb1-12a9b1c04c9e None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:37.108605 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d49851c-9f78-464c-9eb1-12a9b1c04c9e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:37.110001 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d49851c-9f78-464c-9eb1-12a9b1c04c9e None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:37.125003 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 507/1014] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jun 16 15:59:36 2022] POST /identity/v3/services => generated 242 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:59:37.304317 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38639cef-b1b9-49ad-8081-c38a386dadc4 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:37.304850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38639cef-b1b9-49ad-8081-c38a386dadc4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:37.305275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38639cef-b1b9-49ad-8081-c38a386dadc4 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:37.307659 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 508/1015] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:37.314727 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c922f-ef61-4dbc-8fb6-5d309340bc53 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:37.315017 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c922f-ef61-4dbc-8fb6-5d309340bc53 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:37.315489 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c922f-ef61-4dbc-8fb6-5d309340bc53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:37.334167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-c74c922f-ef61-4dbc-8fb6-5d309340bc53 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:37.335711 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-c74c922f-ef61-4dbc-8fb6-5d309340bc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:37.415636 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 508/1016] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:37 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:37.440442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38639cef-b1b9-49ad-8081-c38a386dadc4 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:37.524052 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:37.550291 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38639cef-b1b9-49ad-8081-c38a386dadc4 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:37.587858 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb05f14-4184-4e37-8f21-3a511fc21212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:37.588878 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb05f14-4184-4e37-8f21-3a511fc21212 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:37.589440 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb05f14-4184-4e37-8f21-3a511fc21212 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:37.589885 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb05f14-4184-4e37-8f21-3a511fc21212 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:37.590705 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb05f14-4184-4e37-8f21-3a511fc21212 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:37.593019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb05f14-4184-4e37-8f21-3a511fc21212 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:37.605097 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-abb05f14-4184-4e37-8f21-3a511fc21212 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 15:59:37.605097 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:37.605097 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 15:59:37.605097 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 15:59:37.605097 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 15:59:37.605097 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 15:59:37.605097 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 15:59:37.605097 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:37.605097 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 15:59:37.605097 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:37.605097 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:59:37.605097 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:37.605097 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:37.605097 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:37.605097 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:37.605097 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:37.605097 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:37.606203 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:37.606203 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:37.606203 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:37.606203 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:37.606203 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:37.606203 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:37.606203 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 15:59:37.606203 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 15:59:37.606203 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 15:59:37.606203 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 15:59:37.606203 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:37.606203 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:37.606203 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:37.606203 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:37.606203 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:37.606203 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:37.608211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:37.608211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:37.608211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:37.608211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:37.608211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:37.608211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:37.608211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:37.608211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:37.608211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:37.608211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:37.608211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 15:59:37.608211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:37.608211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 15:59:37.608211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:37.608211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 509/1017] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:37.615408 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c922f-ef61-4dbc-8fb6-5d309340bc53 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:37.718001 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:37.725811 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c922f-ef61-4dbc-8fb6-5d309340bc53 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:37.763521 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daff957-d22a-4758-a73d-00fccd023d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:37.764651 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daff957-d22a-4758-a73d-00fccd023d7c None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:37.764935 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daff957-d22a-4758-a73d-00fccd023d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:37.765210 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daff957-d22a-4758-a73d-00fccd023d7c None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:37.765912 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daff957-d22a-4758-a73d-00fccd023d7c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:37.766360 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daff957-d22a-4758-a73d-00fccd023d7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:37.767620 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daff957-d22a-4758-a73d-00fccd023d7c None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:37.772872 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 509/1018] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jun 16 15:59:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:37.779835 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb05f14-4184-4e37-8f21-3a511fc21212 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:37.888345 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb05f14-4184-4e37-8f21-3a511fc21212 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:37.925611 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc8d71-ea15-4205-b345-33f0180af827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:37.926313 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc8d71-ea15-4205-b345-33f0180af827 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:37.926593 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc8d71-ea15-4205-b345-33f0180af827 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:37.926864 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc8d71-ea15-4205-b345-33f0180af827 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:37.927570 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc8d71-ea15-4205-b345-33f0180af827 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:37.927899 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc8d71-ea15-4205-b345-33f0180af827 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:37.929240 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc8d71-ea15-4205-b345-33f0180af827 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:37.936402 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 510/1019] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jun 16 15:59:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:37.941478 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daff957-d22a-4758-a73d-00fccd023d7c None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:38.049097 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daff957-d22a-4758-a73d-00fccd023d7c None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:38.090486 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee859f0e-9c34-4341-a253-6f74ec660405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:38.091929 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee859f0e-9c34-4341-a253-6f74ec660405 None admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:38.092219 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee859f0e-9c34-4341-a253-6f74ec660405 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:38.092527 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee859f0e-9c34-4341-a253-6f74ec660405 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:38.093233 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee859f0e-9c34-4341-a253-6f74ec660405 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:38.094715 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee859f0e-9c34-4341-a253-6f74ec660405 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:38.122226 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 510/1020] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jun 16 15:59:37 2022] POST /identity/v3/endpoints => generated 334 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:59:38.444418 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e2cae-b18a-4ba6-98c2-abc1a85c2c25 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:38.444941 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e2cae-b18a-4ba6-98c2-abc1a85c2c25 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:38.445020 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e2cae-b18a-4ba6-98c2-abc1a85c2c25 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:38.446566 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 511/1021] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:38.452268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6cbc3-6dfa-4675-bd6f-06d42f877d3a None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:38.452558 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6cbc3-6dfa-4675-bd6f-06d42f877d3a None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:38.452840 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6cbc3-6dfa-4675-bd6f-06d42f877d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:38.470826 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-e2d6cbc3-6dfa-4675-bd6f-06d42f877d3a None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:38.472021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-e2d6cbc3-6dfa-4675-bd6f-06d42f877d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:38.483488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:38.553806 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 511/1022] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:38 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:38.578941 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e2cae-b18a-4ba6-98c2-abc1a85c2c25 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:38.588700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:38.692986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e2cae-b18a-4ba6-98c2-abc1a85c2c25 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:38.733868 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84771f2-88f7-424d-9816-83e6e10744a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:38.735163 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84771f2-88f7-424d-9816-83e6e10744a2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:38.735602 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84771f2-88f7-424d-9816-83e6e10744a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:38.736024 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84771f2-88f7-424d-9816-83e6e10744a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:38.737412 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84771f2-88f7-424d-9816-83e6e10744a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:38.739377 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84771f2-88f7-424d-9816-83e6e10744a2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:38.741802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 512/1023] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:38.747773 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6cbc3-6dfa-4675-bd6f-06d42f877d3a None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:38.860292 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6cbc3-6dfa-4675-bd6f-06d42f877d3a None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:38.896029 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b92b76-6780-43bf-ae3b-6d7179fd1c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:38.896681 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b92b76-6780-43bf-ae3b-6d7179fd1c59 None admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:38.896963 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b92b76-6780-43bf-ae3b-6d7179fd1c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:38.897238 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b92b76-6780-43bf-ae3b-6d7179fd1c59 None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:38.897986 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b92b76-6780-43bf-ae3b-6d7179fd1c59 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:38.899431 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b92b76-6780-43bf-ae3b-6d7179fd1c59 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:38.915919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:38.930463 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 512/1024] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 15:59:38 2022] POST /identity/v3/users => generated 285 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:59:39.376050 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41c7ba-48a7-4c7e-a6d6-b6f44374a53a None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:39.376334 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41c7ba-48a7-4c7e-a6d6-b6f44374a53a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:39.376800 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41c7ba-48a7-4c7e-a6d6-b6f44374a53a None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:39.378349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 513/1025] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:39.383400 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf92e3-1877-4b51-8081-1b11a040cf4f None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:39.383842 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf92e3-1877-4b51-8081-1b11a040cf4f None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:39.384019 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf92e3-1877-4b51-8081-1b11a040cf4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:39.401507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-4fbf92e3-1877-4b51-8081-1b11a040cf4f None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:39.402735 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-4fbf92e3-1877-4b51-8081-1b11a040cf4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:39.481994 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 513/1026] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:39 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:39.504914 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41c7ba-48a7-4c7e-a6d6-b6f44374a53a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:39.585363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:39.616772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41c7ba-48a7-4c7e-a6d6-b6f44374a53a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:39.664732 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2760b-df2c-4eaf-a26c-4ba7ac824772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:39.665450 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2760b-df2c-4eaf-a26c-4ba7ac824772 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:39.665724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2760b-df2c-4eaf-a26c-4ba7ac824772 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:39.666025 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2760b-df2c-4eaf-a26c-4ba7ac824772 None admin] PATH_INFO: `/v3/services/image` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:39.666671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a2760b-df2c-4eaf-a26c-4ba7ac824772 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:39.668038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a2760b-df2c-4eaf-a26c-4ba7ac824772 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:39.680247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-19a2760b-df2c-4eaf-a26c-4ba7ac824772 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 15:59:39.680247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:39.680247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 15:59:39.680247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 15:59:39.680247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 15:59:39.680247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 15:59:39.680247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 15:59:39.680247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:39.680247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 15:59:39.680247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:39.680247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:59:39.680247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:39.680247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:39.680247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:39.680247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:39.680247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:39.680247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:39.681344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:39.681344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:39.681344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:39.681344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:39.681344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:39.681344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:39.681344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 15:59:39.681344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 15:59:39.681344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 15:59:39.681344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 15:59:39.681344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:39.681344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:39.681344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:39.681344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:39.681344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:39.681344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:39.682686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:39.682686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:39.682686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:39.682686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:39.682686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:39.682686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:39.682686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:39.682686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:39.682686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:39.682686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:39.682686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 15:59:39.682686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:39.682686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 15:59:39.682686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:39.683882 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 514/1027] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jun 16 15:59:39 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:39.688136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf92e3-1877-4b51-8081-1b11a040cf4f None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:39.806385 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf92e3-1877-4b51-8081-1b11a040cf4f None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:39.834253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:39.843925 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89679bbe-4781-46e5-b5fa-2878a6513256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:39.844594 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89679bbe-4781-46e5-b5fa-2878a6513256 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:39.844876 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89679bbe-4781-46e5-b5fa-2878a6513256 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:39.845150 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89679bbe-4781-46e5-b5fa-2878a6513256 None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:39.845856 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89679bbe-4781-46e5-b5fa-2878a6513256 None admin] RBAC: Adding query filter params (name=image) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:39.846194 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89679bbe-4781-46e5-b5fa-2878a6513256 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:39.847570 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89679bbe-4781-46e5-b5fa-2878a6513256 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:39.858355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 514/1028] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:39.859640 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2760b-df2c-4eaf-a26c-4ba7ac824772 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:39.970476 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2760b-df2c-4eaf-a26c-4ba7ac824772 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:40.005279 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a7ea7-364d-4c05-8d78-b2b243b9e304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:40.005982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a7ea7-364d-4c05-8d78-b2b243b9e304 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:40.006275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a7ea7-364d-4c05-8d78-b2b243b9e304 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:40.006548 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a7ea7-364d-4c05-8d78-b2b243b9e304 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:40.007262 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a7ea7-364d-4c05-8d78-b2b243b9e304 None admin] RBAC: Adding query filter params (type=image) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:40.007568 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a7ea7-364d-4c05-8d78-b2b243b9e304 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:40.008936 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a7ea7-364d-4c05-8d78-b2b243b9e304 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:40.019769 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 515/1029] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:39 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:40.020736 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89679bbe-4781-46e5-b5fa-2878a6513256 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:40.135745 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89679bbe-4781-46e5-b5fa-2878a6513256 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:40.172584 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11bf183-626b-468f-9904-ba95dda44cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:40.173246 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11bf183-626b-468f-9904-ba95dda44cdf None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:40.173532 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11bf183-626b-468f-9904-ba95dda44cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:40.173804 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11bf183-626b-468f-9904-ba95dda44cdf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:40.174629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11bf183-626b-468f-9904-ba95dda44cdf None admin] RBAC: Adding query filter params (interface=public, service_id=082b3ef5935142859707313ea529c4de, region_id=RegionOne) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:40.174945 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11bf183-626b-468f-9904-ba95dda44cdf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:40.176320 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11bf183-626b-468f-9904-ba95dda44cdf None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:40.186487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 515/1030] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jun 16 15:59:40 2022] GET /identity/v3/endpoints?service_id=082b3ef5935142859707313ea529c4de&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:40.187715 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a7ea7-364d-4c05-8d78-b2b243b9e304 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:40.190625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:40.295758 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a7ea7-364d-4c05-8d78-b2b243b9e304 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:40.330069 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12534a-dec0-4057-a69f-8baaf078697e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:40.330744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12534a-dec0-4057-a69f-8baaf078697e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:40.331028 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12534a-dec0-4057-a69f-8baaf078697e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:40.331307 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12534a-dec0-4057-a69f-8baaf078697e None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:40.331995 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12534a-dec0-4057-a69f-8baaf078697e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:40.333370 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12534a-dec0-4057-a69f-8baaf078697e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:40.346100 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 516/1031] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jun 16 15:59:40 2022] GET /identity/v3/services => generated 1957 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 15:59:40.416939 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacb126-7a1b-403d-8b26-075785e317af None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:40.417472 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacb126-7a1b-403d-8b26-075785e317af None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:40.417472 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacb126-7a1b-403d-8b26-075785e317af None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:40.418974 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 516/1032] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:40.424933 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f4ab8-e0a8-4a72-8377-6b2f90750536 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:40.425231 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f4ab8-e0a8-4a72-8377-6b2f90750536 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:40.425514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f4ab8-e0a8-4a72-8377-6b2f90750536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:40.444391 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-162f4ab8-e0a8-4a72-8377-6b2f90750536 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:40.445623 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-162f4ab8-e0a8-4a72-8377-6b2f90750536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:40.514446 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:40.529950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 517/1033] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:40.550058 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacb126-7a1b-403d-8b26-075785e317af None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:40.612162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:40.663446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacb126-7a1b-403d-8b26-075785e317af None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:40.702169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9cdc9-4584-4bd6-ac86-76b5bd3f39d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:40.702995 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9cdc9-4584-4bd6-ac86-76b5bd3f39d2 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:40.703279 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9cdc9-4584-4bd6-ac86-76b5bd3f39d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:40.703556 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9cdc9-4584-4bd6-ac86-76b5bd3f39d2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:40.714832 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9cdc9-4584-4bd6-ac86-76b5bd3f39d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:40.723718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-0ea9cdc9-4584-4bd6-ac86-76b5bd3f39d2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 15:59:40.723718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:40.723718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:40.723718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:40.723718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:40.723718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:40.723718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:40.723718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:40.723718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:40.723718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:40.723718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:40.723718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:40.723718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:40.723718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:40.723718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:40.723718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:40.724673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:40.724673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:40.724673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:40.724673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:40.724673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:40.724673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:40.724673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:40.724673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:40.724673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:40.724673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:40.724673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:40.724673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:40.724673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:40.724673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:40.724673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:40.724673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:40.725513 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:40.725513 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:40.725513 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:40.725513 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:40.725513 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:40.725513 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:40.725513 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:40.725513 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:40.725513 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:40.725513 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:40.725513 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 15:59:40.725513 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:40.729862 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 517/1034] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jun 16 15:59:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:40.731249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f4ab8-e0a8-4a72-8377-6b2f90750536 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:40.842477 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f4ab8-e0a8-4a72-8377-6b2f90750536 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:40.878793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba39611-82e8-4a80-b9c6-5230023011dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:40.879475 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba39611-82e8-4a80-b9c6-5230023011dc None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:40.879756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba39611-82e8-4a80-b9c6-5230023011dc None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:40.880028 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba39611-82e8-4a80-b9c6-5230023011dc None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:40.880773 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba39611-82e8-4a80-b9c6-5230023011dc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:40.881081 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba39611-82e8-4a80-b9c6-5230023011dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:40.882517 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba39611-82e8-4a80-b9c6-5230023011dc None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:40.896366 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 518/1035] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jun 16 15:59:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:40.898438 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9cdc9-4584-4bd6-ac86-76b5bd3f39d2 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:41.016195 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9cdc9-4584-4bd6-ac86-76b5bd3f39d2 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:41.035936 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:41.061184 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9cd291-3c2d-44a3-917b-e21b57dbfb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:41.062339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cd291-3c2d-44a3-917b-e21b57dbfb53 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:41.062784 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cd291-3c2d-44a3-917b-e21b57dbfb53 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:41.063242 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cd291-3c2d-44a3-917b-e21b57dbfb53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:41.085676 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9cd291-3c2d-44a3-917b-e21b57dbfb53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:41.087573 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9cd291-3c2d-44a3-917b-e21b57dbfb53 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:41.116315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-5a9cd291-3c2d-44a3-917b-e21b57dbfb53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:41.116315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:41.116315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:41.116315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:41.116315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:41.116315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:41.116315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:41.116315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:41.116315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:41.116315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:41.116315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:41.116315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:41.116315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:41.116315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:41.116315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:41.117230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:41.117230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:41.117230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:41.117230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:41.117230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:41.117230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:41.117230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:41.117230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:41.117230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:41.117230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:41.117230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:41.117230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:41.117230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:41.117230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:41.117230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:41.117230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:41.117230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:41.118159 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:41.118159 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:41.118159 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:41.118159 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:41.122334 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 518/1036] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:41.124201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba39611-82e8-4a80-b9c6-5230023011dc None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:41.193949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:41.217863 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb578f01-2c1d-45f2-a9b2-e011ef5223c8 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:41.218207 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb578f01-2c1d-45f2-a9b2-e011ef5223c8 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:41.218450 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb578f01-2c1d-45f2-a9b2-e011ef5223c8 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:41.221289 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 519/1037] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:59:41.226275 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf85ef0-7b29-42b9-b01c-f2bbf219c1ab None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:41.226836 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf85ef0-7b29-42b9-b01c-f2bbf219c1ab None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:41.227336 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf85ef0-7b29-42b9-b01c-f2bbf219c1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:41.247723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-cbf85ef0-7b29-42b9-b01c-f2bbf219c1ab None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:41.249058 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba39611-82e8-4a80-b9c6-5230023011dc None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:41.250066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-cbf85ef0-7b29-42b9-b01c-f2bbf219c1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:41.293302 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa1a75-6915-4522-85f8-044eff34f0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:41.295698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa1a75-6915-4522-85f8-044eff34f0d3 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:41.296210 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa1a75-6915-4522-85f8-044eff34f0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:41.296681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa1a75-6915-4522-85f8-044eff34f0d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:41.297725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faa1a75-6915-4522-85f8-044eff34f0d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:41.298295 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faa1a75-6915-4522-85f8-044eff34f0d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:41.300142 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faa1a75-6915-4522-85f8-044eff34f0d3 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:41.314403 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 519/1038] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:41.320136 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa1a75-6915-4522-85f8-044eff34f0d3 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:41.339319 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 520/1039] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 15:59:41.369766 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf85ef0-7b29-42b9-b01c-f2bbf219c1ab None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:41.478351 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa1a75-6915-4522-85f8-044eff34f0d3 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:41.480074 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:41.485909 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf85ef0-7b29-42b9-b01c-f2bbf219c1ab None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:41.498630 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:41.519145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259a642-10f5-420a-b802-9e5a63af56cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:41.520678 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259a642-10f5-420a-b802-9e5a63af56cf None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:41.520966 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259a642-10f5-420a-b802-9e5a63af56cf None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:41.521244 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259a642-10f5-420a-b802-9e5a63af56cf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:41.529505 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e259a642-10f5-420a-b802-9e5a63af56cf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:41.529782 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf5b454-d17e-4d8e-815e-83c7f58993b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:41.530595 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5b454-d17e-4d8e-815e-83c7f58993b0 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:41.530876 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5b454-d17e-4d8e-815e-83c7f58993b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:41.530990 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e259a642-10f5-420a-b802-9e5a63af56cf None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:41.531147 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5b454-d17e-4d8e-815e-83c7f58993b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:41.542375 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf5b454-d17e-4d8e-815e-83c7f58993b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:41.547171 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-e259a642-10f5-420a-b802-9e5a63af56cf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 15:59:41.547171 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:41.547171 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:41.547171 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:41.547171 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:41.547171 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:41.547171 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:41.547171 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:41.547171 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:41.547171 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:41.547171 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:41.547171 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:41.547171 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:41.547171 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:41.547171 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:41.547171 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:41.548146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:41.548146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:41.548146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:41.548146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:41.548146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:41.548146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:41.548146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:41.548146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:41.548146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:41.548146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:41.548146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:41.548146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:41.548146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:41.548146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:41.548146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:41.548146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:41.548146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:41.548146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:41.554444 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:41.554444 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:41.554444 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:41.554444 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:41.554444 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:41.554444 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:41.554444 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:41.554444 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:41.554444 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 15:59:41.554444 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:41.554444 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 520/1040] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jun 16 15:59:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:41.555164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-abf5b454-d17e-4d8e-815e-83c7f58993b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:59:41.555164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:41.555164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:41.555164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:41.555164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:41.555164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:41.555164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:41.555164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:41.555164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:41.555164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:41.555164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:41.555164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:41.555164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:41.555164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:41.555164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:41.555164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:41.556018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:41.556018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:41.556018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:41.556018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:41.556018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:41.556018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:41.556018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:41.556018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:41.556018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:41.556018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:41.556018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:41.556018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:41.556018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:41.556018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:41.556018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:41.556018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:41.558047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:41.558047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:41.558047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:41.558047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:41.558047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:41.558047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:41.558047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:41.558047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:41.558047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:41.558047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:41.558047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:59:41.558047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:41.558047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 521/1041] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 15:59:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:41.559949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259a642-10f5-420a-b802-9e5a63af56cf None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:41.560098 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf5b454-d17e-4d8e-815e-83c7f58993b0 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:41.670138 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf5b454-d17e-4d8e-815e-83c7f58993b0 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:41.672243 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259a642-10f5-420a-b802-9e5a63af56cf None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:41.705138 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a273c57-3b86-4a34-87e7-4fa801b4ba0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:41.705807 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a273c57-3b86-4a34-87e7-4fa801b4ba0d None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:41.706121 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a273c57-3b86-4a34-87e7-4fa801b4ba0d None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:41.706757 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a273c57-3b86-4a34-87e7-4fa801b4ba0d None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:41.707503 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a273c57-3b86-4a34-87e7-4fa801b4ba0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:41.707815 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a273c57-3b86-4a34-87e7-4fa801b4ba0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:41.709188 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a273c57-3b86-4a34-87e7-4fa801b4ba0d None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:41.716749 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e54397-5823-4576-8e2c-5617ca1f0cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:41.717691 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e54397-5823-4576-8e2c-5617ca1f0cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:41.718223 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e54397-5823-4576-8e2c-5617ca1f0cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:41.718700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e54397-5823-4576-8e2c-5617ca1f0cd2 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:41.719715 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e54397-5823-4576-8e2c-5617ca1f0cd2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:41.720178 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e54397-5823-4576-8e2c-5617ca1f0cd2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:41.721774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e54397-5823-4576-8e2c-5617ca1f0cd2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:41.722285 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 522/1042] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:41.724583 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a273c57-3b86-4a34-87e7-4fa801b4ba0d None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:41.746550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 521/1043] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jun 16 15:59:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:41.750757 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:41.752766 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e54397-5823-4576-8e2c-5617ca1f0cd2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:41.796704 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:41.836690 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a273c57-3b86-4a34-87e7-4fa801b4ba0d None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:41.869948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e54397-5823-4576-8e2c-5617ca1f0cd2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:41.872590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f8be1-ac6a-4bbf-b188-953640dae234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:41.873542 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f8be1-ac6a-4bbf-b188-953640dae234 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:41.873820 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f8be1-ac6a-4bbf-b188-953640dae234 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:41.874124 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f8be1-ac6a-4bbf-b188-953640dae234 None admin] PATH_INFO: `/v3/users/37af54e8e98845179813615798f8d4f7` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:41.888501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53f8be1-ac6a-4bbf-b188-953640dae234 None admin] RBAC: Authorizing `identity:get_user(user_id=37af54e8e98845179813615798f8d4f7)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:41.890173 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53f8be1-ac6a-4bbf-b188-953640dae234 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:41.891889 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 523/1044] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:59:41 2022] GET /identity/v3/users/37af54e8e98845179813615798f8d4f7 => generated 285 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:41.898621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f8be1-ac6a-4bbf-b188-953640dae234 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:41.918210 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1ee1f-2677-4fe8-bc04-7daa79749a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:41.919032 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1ee1f-2677-4fe8-bc04-7daa79749a67 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:41.920038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1ee1f-2677-4fe8-bc04-7daa79749a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:41.920381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1ee1f-2677-4fe8-bc04-7daa79749a67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:41.984633 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:41.998747 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1ee1f-2677-4fe8-bc04-7daa79749a67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:42.000240 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1ee1f-2677-4fe8-bc04-7daa79749a67 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:42.013249 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f8be1-ac6a-4bbf-b188-953640dae234 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:42.042014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:42.054428 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ac243-7d5a-4120-a2fd-2b7eabd990ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:42.055094 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ac243-7d5a-4120-a2fd-2b7eabd990ee None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:42.055383 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ac243-7d5a-4120-a2fd-2b7eabd990ee None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:42.055659 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ac243-7d5a-4120-a2fd-2b7eabd990ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:42.124724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 522/1045] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:41 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 15:59:42.130843 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1ee1f-2677-4fe8-bc04-7daa79749a67 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:42.132495 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ac243-7d5a-4120-a2fd-2b7eabd990ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:42.134004 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ac243-7d5a-4120-a2fd-2b7eabd990ee None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:42.226496 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:42.241720 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1ee1f-2677-4fe8-bc04-7daa79749a67 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:42.257209 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 524/1046] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:41 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 15:59:42.263532 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ac243-7d5a-4120-a2fd-2b7eabd990ee None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:42.280632 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc8309-3dc0-408b-9eae-47daef277cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:42.281430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc8309-3dc0-408b-9eae-47daef277cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:42.281716 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc8309-3dc0-408b-9eae-47daef277cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:42.282037 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc8309-3dc0-408b-9eae-47daef277cb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:42.285119 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:42.295617 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdc8309-3dc0-408b-9eae-47daef277cb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:42.297063 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdc8309-3dc0-408b-9eae-47daef277cb4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:42.315513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-bfdc8309-3dc0-408b-9eae-47daef277cb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:42.315513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:42.315513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:42.315513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:42.315513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:42.315513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:42.315513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:42.315513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:42.315513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:42.315513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:42.315513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:42.315513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:42.315513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:42.315513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:42.315513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:42.316511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:42.316511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:42.316511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:42.316511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:42.316511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:42.316511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:42.316511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:42.316511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:42.316511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:42.316511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:42.316511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:42.316511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:42.316511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:42.316511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:42.316511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:42.316511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:42.316511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:42.317865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:42.317865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:42.317865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:42.317865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:42.317865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 523/1047] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:42.322169 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc8309-3dc0-408b-9eae-47daef277cb4 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:42.378232 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ac243-7d5a-4120-a2fd-2b7eabd990ee None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:42.421234 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74909431-21e3-4376-a7f1-997307ad1aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:42.422863 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74909431-21e3-4376-a7f1-997307ad1aae None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:42.423334 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74909431-21e3-4376-a7f1-997307ad1aae None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:42.423872 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74909431-21e3-4376-a7f1-997307ad1aae None admin] PATH_INFO: `/v3/projects/5bbb8725157b4a0b8e8f02db1f6a3444` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:42.426119 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74909431-21e3-4376-a7f1-997307ad1aae None admin] RBAC: Authorizing `identity:get_project(project_id=5bbb8725157b4a0b8e8f02db1f6a3444)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:42.428553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74909431-21e3-4376-a7f1-997307ad1aae None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:42.430770 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 525/1048] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:59:42 2022] GET /identity/v3/projects/5bbb8725157b4a0b8e8f02db1f6a3444 => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:42.436928 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74909431-21e3-4376-a7f1-997307ad1aae None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:42.444847 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc8309-3dc0-408b-9eae-47daef277cb4 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:42.483028 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d996f7-de9a-4556-b9f9-46c971c8f8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:42.483617 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d996f7-de9a-4556-b9f9-46c971c8f8b8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:42.483904 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d996f7-de9a-4556-b9f9-46c971c8f8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:42.484185 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d996f7-de9a-4556-b9f9-46c971c8f8b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:42.485037 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d996f7-de9a-4556-b9f9-46c971c8f8b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:42.485352 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d996f7-de9a-4556-b9f9-46c971c8f8b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:42.486939 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d996f7-de9a-4556-b9f9-46c971c8f8b8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:42.488200 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:42.501977 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 524/1049] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:42.507794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d996f7-de9a-4556-b9f9-46c971c8f8b8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:42.551144 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74909431-21e3-4376-a7f1-997307ad1aae None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:42.574001 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:42.589412 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88cded-8496-4688-99f4-0ab1c32dd9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:42.590103 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88cded-8496-4688-99f4-0ab1c32dd9d8 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:42.590390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88cded-8496-4688-99f4-0ab1c32dd9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:42.590696 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88cded-8496-4688-99f4-0ab1c32dd9d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:42.591530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88cded-8496-4688-99f4-0ab1c32dd9d8 None admin] RBAC: Adding query filter params (role.id=bbbc77ec645d4ff5a8bf29ce0377f578, scope.project.id=5bbb8725157b4a0b8e8f02db1f6a3444, user.id=37af54e8e98845179813615798f8d4f7) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:42.591844 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88cded-8496-4688-99f4-0ab1c32dd9d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:42.593294 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88cded-8496-4688-99f4-0ab1c32dd9d8 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:42.599056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 526/1050] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:59:42 2022] GET /identity/v3/role_assignments?user.id=37af54e8e98845179813615798f8d4f7&scope.project.id=5bbb8725157b4a0b8e8f02db1f6a3444&role.id=bbbc77ec645d4ff5a8bf29ce0377f578 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:42.609354 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7e17e-cade-4327-a9a5-ea7b5a7c7954 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:42.609741 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7e17e-cade-4327-a9a5-ea7b5a7c7954 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:42.610033 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7e17e-cade-4327-a9a5-ea7b5a7c7954 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:42.611701 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 527/1051] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:42.617576 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514044d-2b6a-47cb-b9b2-4bf6d44b3b87 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:42.617907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514044d-2b6a-47cb-b9b2-4bf6d44b3b87 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:42.618175 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514044d-2b6a-47cb-b9b2-4bf6d44b3b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:42.641133 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-9514044d-2b6a-47cb-b9b2-4bf6d44b3b87 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:42.642450 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-9514044d-2b6a-47cb-b9b2-4bf6d44b3b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:42.646430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d996f7-de9a-4556-b9f9-46c971c8f8b8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:42.688544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e785e8a-854c-4fd4-a121-b1da7c92c943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:42.689792 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e785e8a-854c-4fd4-a121-b1da7c92c943 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:42.689792 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e785e8a-854c-4fd4-a121-b1da7c92c943 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:42.690172 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e785e8a-854c-4fd4-a121-b1da7c92c943 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:42.698614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e785e8a-854c-4fd4-a121-b1da7c92c943 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:42.700183 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e785e8a-854c-4fd4-a121-b1da7c92c943 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:42.715062 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-6e785e8a-854c-4fd4-a121-b1da7c92c943 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:42.715062 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:42.715062 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:42.715062 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:42.715062 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:42.715062 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:42.715062 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:42.715062 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:42.715062 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:42.715062 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:42.715062 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:42.715062 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:42.715062 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:42.715062 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:42.715062 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:42.716523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:42.716523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:42.716523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:42.716523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:42.716523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:42.716523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:42.716523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:42.716523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:42.716523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:42.716523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:42.716523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:42.716523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:42.716523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:42.716523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:42.716523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:42.716523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:42.716523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:42.717793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:42.717793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:42.717793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:42.717793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:42.717793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:42.717793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:42.717793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:42.717793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:42.717793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 525/1052] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:42.722699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e785e8a-854c-4fd4-a121-b1da7c92c943 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:42.734064 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 528/1053] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 15:59:42.755918 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514044d-2b6a-47cb-b9b2-4bf6d44b3b87 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:42.807108 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:42.845937 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e785e8a-854c-4fd4-a121-b1da7c92c943 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:42.855999 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:42.870580 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514044d-2b6a-47cb-b9b2-4bf6d44b3b87 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:42.886919 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a9745-23b1-49ea-ba32-ad9dd8523453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:42.888635 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a9745-23b1-49ea-ba32-ad9dd8523453 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:42.888913 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a9745-23b1-49ea-ba32-ad9dd8523453 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:42.889186 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a9745-23b1-49ea-ba32-ad9dd8523453 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:42.889997 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a9745-23b1-49ea-ba32-ad9dd8523453 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:42.890307 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a9745-23b1-49ea-ba32-ad9dd8523453 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:42.891753 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a9745-23b1-49ea-ba32-ad9dd8523453 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:42.905972 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 526/1054] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:42.911068 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358107a0-4002-4c93-af23-d40c2df096b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:42.911877 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358107a0-4002-4c93-af23-d40c2df096b4 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:42.912162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358107a0-4002-4c93-af23-d40c2df096b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:42.912437 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358107a0-4002-4c93-af23-d40c2df096b4 None admin] PATH_INFO: `/v3/services/image` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:42.912704 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a9745-23b1-49ea-ba32-ad9dd8523453 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:42.913094 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358107a0-4002-4c93-af23-d40c2df096b4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:42.914521 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358107a0-4002-4c93-af23-d40c2df096b4 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:42.928800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-358107a0-4002-4c93-af23-d40c2df096b4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 15:59:42.928800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:42.928800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 15:59:42.928800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 15:59:42.928800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 15:59:42.928800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 15:59:42.928800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 15:59:42.928800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:42.928800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 15:59:42.928800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:42.928800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:59:42.928800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:42.928800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:42.928800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:42.928800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:42.928800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:42.928800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:42.929845 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:42.929845 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:42.929845 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:42.929845 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:42.929845 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:42.929845 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:42.929845 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 15:59:42.929845 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 15:59:42.929845 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 15:59:42.929845 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 15:59:42.929845 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:42.929845 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:42.929845 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:42.929845 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:42.929845 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:42.929845 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:42.930783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:42.930783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:42.930783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:42.930783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:42.930783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:42.930783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:42.930783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:42.930783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:42.930783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:42.930783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:42.930783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 15:59:42.930783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:42.930783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 15:59:42.930783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:42.930783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 529/1055] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jun 16 15:59:42 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:42.936787 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358107a0-4002-4c93-af23-d40c2df096b4 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:43.035137 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a9745-23b1-49ea-ba32-ad9dd8523453 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:43.047979 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358107a0-4002-4c93-af23-d40c2df096b4 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:43.051621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:43.072799 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e04737c-fd32-4bfb-8c44-316171b918b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:43.073462 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04737c-fd32-4bfb-8c44-316171b918b0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:43.073744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04737c-fd32-4bfb-8c44-316171b918b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:43.074046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04737c-fd32-4bfb-8c44-316171b918b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:43.074876 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e04737c-fd32-4bfb-8c44-316171b918b0 None admin] RBAC: Adding query filter params (role.id=758069bda1404a3589771a8f43e95187, scope.project.id=b1177a85a0de4276898daf4a24ff069f, user.id=c3645b1a0be34c01ac87c734fcd85ff6) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:43.075241 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e04737c-fd32-4bfb-8c44-316171b918b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:43.076839 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e04737c-fd32-4bfb-8c44-316171b918b0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:43.085999 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 527/1056] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:59:42 2022] GET /identity/v3/role_assignments?user.id=c3645b1a0be34c01ac87c734fcd85ff6&scope.project.id=b1177a85a0de4276898daf4a24ff069f&role.id=758069bda1404a3589771a8f43e95187 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:43.097390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9717b6-b588-465d-a38f-8e27d7aae11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:43.098131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9717b6-b588-465d-a38f-8e27d7aae11e None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:43.098418 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9717b6-b588-465d-a38f-8e27d7aae11e None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:43.098696 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9717b6-b588-465d-a38f-8e27d7aae11e None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:43.099432 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9717b6-b588-465d-a38f-8e27d7aae11e None admin] RBAC: Adding query filter params (name=image) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:43.099746 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9717b6-b588-465d-a38f-8e27d7aae11e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:43.101192 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9717b6-b588-465d-a38f-8e27d7aae11e None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:43.112593 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 530/1057] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:43.113957 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e04737c-fd32-4bfb-8c44-316171b918b0 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:43.150264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:43.230270 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e04737c-fd32-4bfb-8c44-316171b918b0 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:43.269884 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f910ec-00c3-459f-80e0-9f8fcaac148e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:43.270580 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f910ec-00c3-459f-80e0-9f8fcaac148e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:43.270861 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f910ec-00c3-459f-80e0-9f8fcaac148e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:43.271186 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f910ec-00c3-459f-80e0-9f8fcaac148e None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:43.271875 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f910ec-00c3-459f-80e0-9f8fcaac148e None admin] RBAC: Adding query filter params (type=image) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:43.272177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f910ec-00c3-459f-80e0-9f8fcaac148e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:43.273542 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f910ec-00c3-459f-80e0-9f8fcaac148e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:43.279166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 528/1058] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:43.285043 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9717b6-b588-465d-a38f-8e27d7aae11e None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:43.393715 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9717b6-b588-465d-a38f-8e27d7aae11e None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:43.431063 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e30753-a46d-42fd-8c95-fdf5baf748a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:43.431743 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e30753-a46d-42fd-8c95-fdf5baf748a6 None admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:43.432051 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e30753-a46d-42fd-8c95-fdf5baf748a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:43.432355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e30753-a46d-42fd-8c95-fdf5baf748a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:43.433143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e30753-a46d-42fd-8c95-fdf5baf748a6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:43.435501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e30753-a46d-42fd-8c95-fdf5baf748a6 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:43.466401 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 531/1059] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jun 16 15:59:43 2022] POST /identity/v3/endpoints => generated 327 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:59:44.768978 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4ae0f-da34-4c64-80ca-785d41fe070c None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:44.769513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4ae0f-da34-4c64-80ca-785d41fe070c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:44.769513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4ae0f-da34-4c64-80ca-785d41fe070c None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:44.771115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 529/1060] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:44.776405 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b13fa3-0f48-4c0f-a33a-08c4ddeff25a None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:44.776846 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b13fa3-0f48-4c0f-a33a-08c4ddeff25a None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:44.777035 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b13fa3-0f48-4c0f-a33a-08c4ddeff25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:44.798842 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-77b13fa3-0f48-4c0f-a33a-08c4ddeff25a None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:44.800058 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-77b13fa3-0f48-4c0f-a33a-08c4ddeff25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:44.820549 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:44.883769 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 532/1061] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:44.906677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4ae0f-da34-4c64-80ca-785d41fe070c None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:45.013482 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:45.021941 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4ae0f-da34-4c64-80ca-785d41fe070c None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:45.061694 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87960bc-49a4-4771-bff4-bb1c9119ac37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:45.062580 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87960bc-49a4-4771-bff4-bb1c9119ac37 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:45.062960 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87960bc-49a4-4771-bff4-bb1c9119ac37 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:45.063264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87960bc-49a4-4771-bff4-bb1c9119ac37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:45.076601 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87960bc-49a4-4771-bff4-bb1c9119ac37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:45.089703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-f87960bc-49a4-4771-bff4-bb1c9119ac37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:59:45.089703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:45.089703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:45.089703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:45.089703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:45.089703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:45.089703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:45.089703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:45.089703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:45.089703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:45.089703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:45.089703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:45.089703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:45.089703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:45.089703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:45.089703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:45.090909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:45.090909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:45.090909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:45.090909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:45.090909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:45.090909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:45.090909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:45.090909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:45.090909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:45.090909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:45.090909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:45.090909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:45.090909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:45.090909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:45.090909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:45.090909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:45.091761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:45.091761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:45.091761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:45.091761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:45.091761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:45.091761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:45.091761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:45.091761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:45.091761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:45.091761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:45.091761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:59:45.091761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:45.094307 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 530/1062] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 15:59:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:45.097614 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b13fa3-0f48-4c0f-a33a-08c4ddeff25a None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:45.211801 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b13fa3-0f48-4c0f-a33a-08c4ddeff25a None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:45.248450 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29011ea-d59e-4877-969e-1555735806c2 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:45.248766 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29011ea-d59e-4877-969e-1555735806c2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:45.249019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29011ea-d59e-4877-969e-1555735806c2 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:45.249727 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be6b3b-25a7-4184-a55e-11ba34536eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:45.250482 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be6b3b-25a7-4184-a55e-11ba34536eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:45.250772 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be6b3b-25a7-4184-a55e-11ba34536eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:45.250889 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 531/1063] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:45.251110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be6b3b-25a7-4184-a55e-11ba34536eb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:45.251881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be6b3b-25a7-4184-a55e-11ba34536eb6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:45.252252 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be6b3b-25a7-4184-a55e-11ba34536eb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:45.253644 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be6b3b-25a7-4184-a55e-11ba34536eb6 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:45.256222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540332f-d337-4a4c-b31d-43d093d2a0ee None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:45.256512 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540332f-d337-4a4c-b31d-43d093d2a0ee None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:45.256790 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540332f-d337-4a4c-b31d-43d093d2a0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:45.265251 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 533/1064] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:45.272631 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be6b3b-25a7-4184-a55e-11ba34536eb6 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:45.278126 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-6540332f-d337-4a4c-b31d-43d093d2a0ee None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:45.279340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6540332f-d337-4a4c-b31d-43d093d2a0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:45.289580 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:45.359114 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 532/1065] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:45.382064 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540332f-d337-4a4c-b31d-43d093d2a0ee None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:45.386789 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be6b3b-25a7-4184-a55e-11ba34536eb6 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:45.425058 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4e12f-32ef-439a-9a00-8e21c74abb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:45.425981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4e12f-32ef-439a-9a00-8e21c74abb13 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:45.426264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4e12f-32ef-439a-9a00-8e21c74abb13 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:45.426541 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4e12f-32ef-439a-9a00-8e21c74abb13 None admin] PATH_INFO: `/v3/users/37af54e8e98845179813615798f8d4f7` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:45.428025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc4e12f-32ef-439a-9a00-8e21c74abb13 None admin] RBAC: Authorizing `identity:get_user(user_id=37af54e8e98845179813615798f8d4f7)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:45.429436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc4e12f-32ef-439a-9a00-8e21c74abb13 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:45.431135 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 534/1066] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:59:45 2022] GET /identity/v3/users/37af54e8e98845179813615798f8d4f7 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:45.437682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4e12f-32ef-439a-9a00-8e21c74abb13 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:45.474016 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:45.490332 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540332f-d337-4a4c-b31d-43d093d2a0ee None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:45.527044 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec19e62-5f5f-4b9c-9f74-abefe525a239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:45.527781 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec19e62-5f5f-4b9c-9f74-abefe525a239 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:45.528094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec19e62-5f5f-4b9c-9f74-abefe525a239 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:45.528375 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec19e62-5f5f-4b9c-9f74-abefe525a239 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:45.536959 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:45.539621 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec19e62-5f5f-4b9c-9f74-abefe525a239 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:45.548434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-fec19e62-5f5f-4b9c-9f74-abefe525a239 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 15:59:45.548434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:45.548434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:45.548434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:45.548434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:45.548434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:45.548434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:45.548434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:45.548434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:45.548434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:45.548434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:45.548434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:45.548434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:45.548434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:45.548434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:45.548434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:45.549414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:45.549414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:45.549414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:45.549414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:45.549414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:45.549414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:45.549414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:45.549414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:45.549414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:45.549414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:45.549414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:45.549414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:45.549414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:45.549414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:45.549414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:45.549414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:45.550339 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:45.550339 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:45.550339 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:45.550339 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:45.550339 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:45.550339 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:45.550339 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:45.550339 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:45.550339 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:45.550339 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:45.550339 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 15:59:45.550339 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:45.550339 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 533/1067] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jun 16 15:59:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:45.556146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec19e62-5f5f-4b9c-9f74-abefe525a239 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:45.562913 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4e12f-32ef-439a-9a00-8e21c74abb13 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:45.601714 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d9526-1f81-404c-a0ec-d0404036ea6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:45.602630 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d9526-1f81-404c-a0ec-d0404036ea6d None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:45.602935 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d9526-1f81-404c-a0ec-d0404036ea6d None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:45.603195 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d9526-1f81-404c-a0ec-d0404036ea6d None admin] PATH_INFO: `/v3/projects/5bbb8725157b4a0b8e8f02db1f6a3444` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:45.604582 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d9526-1f81-404c-a0ec-d0404036ea6d None admin] RBAC: Authorizing `identity:get_project(project_id=5bbb8725157b4a0b8e8f02db1f6a3444)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:45.606088 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d9526-1f81-404c-a0ec-d0404036ea6d None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:45.608105 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 535/1068] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:59:45 2022] GET /identity/v3/projects/5bbb8725157b4a0b8e8f02db1f6a3444 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:45.614170 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d9526-1f81-404c-a0ec-d0404036ea6d None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:45.677793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec19e62-5f5f-4b9c-9f74-abefe525a239 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:45.713692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1f1fd-bcea-4152-b65c-b99e1abc59d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:45.714403 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1f1fd-bcea-4152-b65c-b99e1abc59d0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:45.714689 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1f1fd-bcea-4152-b65c-b99e1abc59d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:45.714970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1f1fd-bcea-4152-b65c-b99e1abc59d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:45.715757 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1f1fd-bcea-4152-b65c-b99e1abc59d0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:45.716060 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1f1fd-bcea-4152-b65c-b99e1abc59d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:45.717436 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1f1fd-bcea-4152-b65c-b99e1abc59d0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:45.718287 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d9526-1f81-404c-a0ec-d0404036ea6d None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:45.725800 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 534/1069] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jun 16 15:59:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:45.727143 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a149f8-e705-4674-991a-8115bc878270 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:45.727432 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a149f8-e705-4674-991a-8115bc878270 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:45.727715 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a149f8-e705-4674-991a-8115bc878270 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:45.729151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 535/1070] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:45.734696 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a149f8-e705-4674-991a-8115bc878270 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:45.746542 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:45.758453 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e091d-7e63-4a36-a1bf-090d4af8fd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:45.759601 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e091d-7e63-4a36-a1bf-090d4af8fd8d None admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:45.759885 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e091d-7e63-4a36-a1bf-090d4af8fd8d None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:45.760155 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e091d-7e63-4a36-a1bf-090d4af8fd8d None admin] PATH_INFO: `/v3/projects/5bbb8725157b4a0b8e8f02db1f6a3444/users/37af54e8e98845179813615798f8d4f7/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:45.762182 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e091d-7e63-4a36-a1bf-090d4af8fd8d None admin] RBAC: Authorizing `identity:create_grant(project_id=5bbb8725157b4a0b8e8f02db1f6a3444, user_id=37af54e8e98845179813615798f8d4f7, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:45.763765 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e091d-7e63-4a36-a1bf-090d4af8fd8d None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:45.777823 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 536/1071] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 15:59:45 2022] PUT /identity/v3/projects/5bbb8725157b4a0b8e8f02db1f6a3444/users/37af54e8e98845179813615798f8d4f7/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:59:45.779801 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fde876-6682-48e1-8218-3f72b60c0cbc None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:45.780094 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fde876-6682-48e1-8218-3f72b60c0cbc None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:45.780375 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fde876-6682-48e1-8218-3f72b60c0cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:45.812859 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-f4fde876-6682-48e1-8218-3f72b60c0cbc None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:45.814260 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-f4fde876-6682-48e1-8218-3f72b60c0cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:45.816715 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:45.868540 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a149f8-e705-4674-991a-8115bc878270 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:45.904919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 537/1072] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:45.913165 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04844d2a-0d05-4a5f-91f5-3d7f088107ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:45.913995 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04844d2a-0d05-4a5f-91f5-3d7f088107ad None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:45.914275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04844d2a-0d05-4a5f-91f5-3d7f088107ad None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:45.914586 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04844d2a-0d05-4a5f-91f5-3d7f088107ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:45.928593 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fde876-6682-48e1-8218-3f72b60c0cbc None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:45.928797 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04844d2a-0d05-4a5f-91f5-3d7f088107ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:45.930255 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04844d2a-0d05-4a5f-91f5-3d7f088107ad None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:45.956207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-04844d2a-0d05-4a5f-91f5-3d7f088107ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:45.956207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:45.956207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:45.956207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:45.956207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:45.956207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:45.956207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:45.956207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:45.956207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:45.956207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:45.956207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:45.956207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:45.956207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:45.956207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:45.956207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:45.957461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:45.957461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:45.957461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:45.957461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:45.957461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:45.957461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:45.957461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:45.957461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:45.957461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:45.957461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:45.957461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:45.957461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:45.957461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:45.957461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:45.957461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:45.957461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:45.957461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:45.958704 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:45.958704 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:45.958704 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:45.958704 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:45.960287 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 536/1073] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:45.966685 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04844d2a-0d05-4a5f-91f5-3d7f088107ad None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:46.034999 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:46.040309 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fde876-6682-48e1-8218-3f72b60c0cbc None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:46.079237 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff992966-a2ab-444f-ba2e-eafb466869e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:46.079895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff992966-a2ab-444f-ba2e-eafb466869e4 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:46.080175 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff992966-a2ab-444f-ba2e-eafb466869e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:46.080455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff992966-a2ab-444f-ba2e-eafb466869e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:46.083403 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04844d2a-0d05-4a5f-91f5-3d7f088107ad None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:46.088820 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:46.123920 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190d134-0bc9-4c4b-af20-d589c8d7428e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:46.124604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190d134-0bc9-4c4b-af20-d589c8d7428e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:46.124947 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190d134-0bc9-4c4b-af20-d589c8d7428e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:46.125233 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190d134-0bc9-4c4b-af20-d589c8d7428e None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:46.125995 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190d134-0bc9-4c4b-af20-d589c8d7428e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:46.126311 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190d134-0bc9-4c4b-af20-d589c8d7428e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:46.127942 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190d134-0bc9-4c4b-af20-d589c8d7428e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:46.146362 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 537/1074] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:46.147794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190d134-0bc9-4c4b-af20-d589c8d7428e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:46.156386 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff992966-a2ab-444f-ba2e-eafb466869e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:46.157893 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff992966-a2ab-444f-ba2e-eafb466869e4 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:46.254042 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190d134-0bc9-4c4b-af20-d589c8d7428e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:46.275573 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:46.287389 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 538/1075] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 15:59:46.288234 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff992966-a2ab-444f-ba2e-eafb466869e4 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:46.291672 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add14ed-a9ac-44bf-a91e-941e536ddbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:46.292484 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add14ed-a9ac-44bf-a91e-941e536ddbff None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:46.292769 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add14ed-a9ac-44bf-a91e-941e536ddbff None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:46.293053 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add14ed-a9ac-44bf-a91e-941e536ddbff None admin] PATH_INFO: `/v3/users/nova` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:46.301583 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add14ed-a9ac-44bf-a91e-941e536ddbff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:46.303079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add14ed-a9ac-44bf-a91e-941e536ddbff None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:46.317680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-1add14ed-a9ac-44bf-a91e-941e536ddbff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 15:59:46.317680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:46.317680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:46.317680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:46.317680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:46.317680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:46.317680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:46.317680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:46.317680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:46.317680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:46.317680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:46.317680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:46.317680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:46.317680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:46.317680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:46.317680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:46.318838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:46.318838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:46.318838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:46.318838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:46.318838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:46.318838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:46.318838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:46.318838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:46.318838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:46.318838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:46.318838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:46.318838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:46.318838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:46.318838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:46.318838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:46.318838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:46.318838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:46.318838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:46.319784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:46.319784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:46.319784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:46.319784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:46.319784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:46.319784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:46.319784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:46.319784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:46.319784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 15:59:46.319784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:46.319784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 538/1076] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jun 16 15:59:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:46.326368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add14ed-a9ac-44bf-a91e-941e536ddbff None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:46.338044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:46.404794 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff992966-a2ab-444f-ba2e-eafb466869e4 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:46.442168 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3b06b-1aa2-4efc-af06-eb050567f655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:46.442872 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3b06b-1aa2-4efc-af06-eb050567f655 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:46.443156 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3b06b-1aa2-4efc-af06-eb050567f655 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:46.443436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3b06b-1aa2-4efc-af06-eb050567f655 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:46.447892 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add14ed-a9ac-44bf-a91e-941e536ddbff None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:46.455175 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea3b06b-1aa2-4efc-af06-eb050567f655 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:46.456665 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea3b06b-1aa2-4efc-af06-eb050567f655 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:46.476562 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-9ea3b06b-1aa2-4efc-af06-eb050567f655 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:46.476562 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:46.476562 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:46.476562 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:46.476562 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:46.476562 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:46.476562 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:46.476562 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:46.476562 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:46.476562 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:46.476562 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:46.476562 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:46.476562 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:46.476562 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:46.476562 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:46.477629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:46.477629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:46.477629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:46.477629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:46.477629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:46.477629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:46.477629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:46.477629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:46.477629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:46.477629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:46.477629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:46.477629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:46.477629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:46.477629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:46.477629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:46.477629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:46.477629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:46.478649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:46.478649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:46.478649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:46.478649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:46.480274 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 539/1077] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:46.487232 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3b06b-1aa2-4efc-af06-eb050567f655 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:46.492424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5bedf2-cae6-4eaf-a9da-69f33897c994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:46.493111 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5bedf2-cae6-4eaf-a9da-69f33897c994 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:46.493393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5bedf2-cae6-4eaf-a9da-69f33897c994 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:46.493667 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5bedf2-cae6-4eaf-a9da-69f33897c994 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:46.494543 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5bedf2-cae6-4eaf-a9da-69f33897c994 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:46.494855 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5bedf2-cae6-4eaf-a9da-69f33897c994 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:46.496295 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5bedf2-cae6-4eaf-a9da-69f33897c994 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:46.515065 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 539/1078] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jun 16 15:59:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:46.521251 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5bedf2-cae6-4eaf-a9da-69f33897c994 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:46.564261 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:46.603377 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:46.616590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3b06b-1aa2-4efc-af06-eb050567f655 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:46.634025 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5bedf2-cae6-4eaf-a9da-69f33897c994 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:46.654415 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39bf6b-72b1-4c72-87d6-9adf5dcad904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:46.655080 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39bf6b-72b1-4c72-87d6-9adf5dcad904 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:46.655399 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39bf6b-72b1-4c72-87d6-9adf5dcad904 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:46.655814 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39bf6b-72b1-4c72-87d6-9adf5dcad904 None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:46.656949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed39bf6b-72b1-4c72-87d6-9adf5dcad904 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:46.657408 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed39bf6b-72b1-4c72-87d6-9adf5dcad904 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:46.659644 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed39bf6b-72b1-4c72-87d6-9adf5dcad904 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:46.672721 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821f1e1-1dcb-4ff2-8d43-50fa9ce2ad1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:46.682021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 540/1079] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:46.682247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821f1e1-1dcb-4ff2-8d43-50fa9ce2ad1c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:46.682247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821f1e1-1dcb-4ff2-8d43-50fa9ce2ad1c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:46.682247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821f1e1-1dcb-4ff2-8d43-50fa9ce2ad1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:46.693241 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5821f1e1-1dcb-4ff2-8d43-50fa9ce2ad1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:46.694912 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5821f1e1-1dcb-4ff2-8d43-50fa9ce2ad1c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:46.716770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-5821f1e1-1dcb-4ff2-8d43-50fa9ce2ad1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:46.716770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:46.716770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:46.716770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:46.716770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:46.716770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:46.716770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:46.716770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:46.716770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:46.716770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:46.716770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:46.716770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:46.716770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:46.716770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:46.716770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:46.717772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:46.717772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:46.717772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:46.717772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:46.717772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:46.717772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:46.717772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:46.717772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:46.717772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:46.717772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:46.717772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:46.717772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:46.717772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:46.717772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:46.717772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:46.717772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:46.717772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:46.719060 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:46.719060 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:46.719060 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:46.719060 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:46.719060 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 540/1080] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:46.724117 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39bf6b-72b1-4c72-87d6-9adf5dcad904 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:46.850675 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39bf6b-72b1-4c72-87d6-9adf5dcad904 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:46.893018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b01fd-541f-449f-b34e-63cc8bd38dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:46.893693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b01fd-541f-449f-b34e-63cc8bd38dbd None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:46.894005 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b01fd-541f-449f-b34e-63cc8bd38dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:46.894283 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b01fd-541f-449f-b34e-63cc8bd38dbd None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:46.895009 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b01fd-541f-449f-b34e-63cc8bd38dbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:46.895320 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b01fd-541f-449f-b34e-63cc8bd38dbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:46.897222 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b01fd-541f-449f-b34e-63cc8bd38dbd None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:46.910145 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 541/1081] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:46.916176 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821f1e1-1dcb-4ff2-8d43-50fa9ce2ad1c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:47.020214 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:47.031910 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821f1e1-1dcb-4ff2-8d43-50fa9ce2ad1c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:47.066584 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99131531-9d4d-4ded-b173-ac8e73bd8db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:47.067382 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99131531-9d4d-4ded-b173-ac8e73bd8db4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:47.067666 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99131531-9d4d-4ded-b173-ac8e73bd8db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:47.067941 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99131531-9d4d-4ded-b173-ac8e73bd8db4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:47.074632 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99131531-9d4d-4ded-b173-ac8e73bd8db4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:47.076563 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99131531-9d4d-4ded-b173-ac8e73bd8db4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:47.091421 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-99131531-9d4d-4ded-b173-ac8e73bd8db4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:47.091421 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:47.091421 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:47.091421 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:47.091421 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:47.091421 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:47.091421 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:47.091421 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:47.091421 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:47.091421 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:47.091421 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:47.091421 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:47.091421 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:47.091421 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:47.091421 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:47.092379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:47.092379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:47.092379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:47.092379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:47.092379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:47.092379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:47.092379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:47.092379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:47.092379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:47.092379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:47.092379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:47.092379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:47.092379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:47.092379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:47.092379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:47.092379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:47.092379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:47.093276 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:47.093276 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:47.093276 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:47.093276 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:47.093276 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:47.093276 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:47.093276 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:47.093276 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:47.093987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 541/1082] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:47.098652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b01fd-541f-449f-b34e-63cc8bd38dbd None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:47.113184 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:47.202200 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b01fd-541f-449f-b34e-63cc8bd38dbd None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:47.236200 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9e59d-bb20-4dfd-a5d5-614e65101b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:47.236853 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9e59d-bb20-4dfd-a5d5-614e65101b49 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:47.237136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9e59d-bb20-4dfd-a5d5-614e65101b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:47.237406 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9e59d-bb20-4dfd-a5d5-614e65101b49 None admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:47.238220 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd9e59d-bb20-4dfd-a5d5-614e65101b49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:47.238530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd9e59d-bb20-4dfd-a5d5-614e65101b49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:47.239963 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd9e59d-bb20-4dfd-a5d5-614e65101b49 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:47.252996 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 542/1083] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:47.259625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99131531-9d4d-4ded-b173-ac8e73bd8db4 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:47.361389 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99131531-9d4d-4ded-b173-ac8e73bd8db4 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:47.395789 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55fc71-8b7e-4b34-9abe-277c89c9240c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:47.396601 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55fc71-8b7e-4b34-9abe-277c89c9240c None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:47.396888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55fc71-8b7e-4b34-9abe-277c89c9240c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:47.397164 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55fc71-8b7e-4b34-9abe-277c89c9240c None admin] PATH_INFO: `/v3/projects/b1177a85a0de4276898daf4a24ff069f/users/c3645b1a0be34c01ac87c734fcd85ff6/roles/758069bda1404a3589771a8f43e95187` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:47.399411 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55fc71-8b7e-4b34-9abe-277c89c9240c None admin] RBAC: Authorizing `identity:create_grant(project_id=b1177a85a0de4276898daf4a24ff069f, user_id=c3645b1a0be34c01ac87c734fcd85ff6, role_id=758069bda1404a3589771a8f43e95187)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:47.400850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55fc71-8b7e-4b34-9abe-277c89c9240c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:47.414589 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 542/1084] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 15:59:47 2022] PUT /identity/v3/projects/b1177a85a0de4276898daf4a24ff069f/users/c3645b1a0be34c01ac87c734fcd85ff6/roles/758069bda1404a3589771a8f43e95187 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:59:47.944374 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d954f58-696d-4f1e-a468-fae4ab8b50c6 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:47.944901 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d954f58-696d-4f1e-a468-fae4ab8b50c6 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:47.945025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d954f58-696d-4f1e-a468-fae4ab8b50c6 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:47.946418 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 543/1085] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:47.952479 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe69e8-8771-4b3d-a904-4d6e38327ac8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:47.952884 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe69e8-8771-4b3d-a904-4d6e38327ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:47.953159 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe69e8-8771-4b3d-a904-4d6e38327ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:47.970519 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8bbe69e8-8771-4b3d-a904-4d6e38327ac8 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:47.971700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8bbe69e8-8771-4b3d-a904-4d6e38327ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:47.974536 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:48.043952 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 543/1086] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:48.068321 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d954f58-696d-4f1e-a468-fae4ab8b50c6 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:48.149730 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:48.181900 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d954f58-696d-4f1e-a468-fae4ab8b50c6 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:48.215991 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855ca81-821d-4117-9ad9-713041bd8f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:48.216786 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855ca81-821d-4117-9ad9-713041bd8f09 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:48.217065 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855ca81-821d-4117-9ad9-713041bd8f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:48.217332 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855ca81-821d-4117-9ad9-713041bd8f09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:48.228207 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855ca81-821d-4117-9ad9-713041bd8f09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:48.235691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-1855ca81-821d-4117-9ad9-713041bd8f09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:59:48.235691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:48.235691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:48.235691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:48.235691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:48.235691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:48.235691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:48.235691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:48.235691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:48.235691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:48.235691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:48.235691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:48.235691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:48.235691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:48.235691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:48.235691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:48.236883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:48.236883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:48.236883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:48.236883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:48.236883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:48.236883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:48.236883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:48.236883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:48.236883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:48.236883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:48.236883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:48.236883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:48.236883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:48.236883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:48.236883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:48.236883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:48.238208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:48.238208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:48.238208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:48.238208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:48.238208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:48.238208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:48.238208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:48.238208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:48.238208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:48.238208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:48.238208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:59:48.238208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:48.238208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 544/1087] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 15:59:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:48.242797 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe69e8-8771-4b3d-a904-4d6e38327ac8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:48.343899 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe69e8-8771-4b3d-a904-4d6e38327ac8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:48.378892 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5297b5cd-3ee2-4bd1-9865-3e8d37630d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:48.379558 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297b5cd-3ee2-4bd1-9865-3e8d37630d72 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:48.379845 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297b5cd-3ee2-4bd1-9865-3e8d37630d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:48.380115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297b5cd-3ee2-4bd1-9865-3e8d37630d72 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:48.380849 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5297b5cd-3ee2-4bd1-9865-3e8d37630d72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:48.381147 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5297b5cd-3ee2-4bd1-9865-3e8d37630d72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:48.382542 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5297b5cd-3ee2-4bd1-9865-3e8d37630d72 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:48.391802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 544/1088] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:48.398488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855ca81-821d-4117-9ad9-713041bd8f09 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:48.514419 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855ca81-821d-4117-9ad9-713041bd8f09 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:48.549217 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:48.556251 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96c5ea-36b8-40d1-a385-57f11acf49cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:48.557120 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96c5ea-36b8-40d1-a385-57f11acf49cc None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:48.557400 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96c5ea-36b8-40d1-a385-57f11acf49cc None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:48.557672 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96c5ea-36b8-40d1-a385-57f11acf49cc None admin] PATH_INFO: `/v3/users/37af54e8e98845179813615798f8d4f7` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:48.558899 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96c5ea-36b8-40d1-a385-57f11acf49cc None admin] RBAC: Authorizing `identity:get_user(user_id=37af54e8e98845179813615798f8d4f7)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:48.561747 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96c5ea-36b8-40d1-a385-57f11acf49cc None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:48.563447 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 545/1089] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:59:48 2022] GET /identity/v3/users/37af54e8e98845179813615798f8d4f7 => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:48.569686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5297b5cd-3ee2-4bd1-9865-3e8d37630d72 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:48.573422 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:48.674643 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5297b5cd-3ee2-4bd1-9865-3e8d37630d72 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:48.710042 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1975c71-282a-4480-afcb-0a5e56a925e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:48.710685 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1975c71-282a-4480-afcb-0a5e56a925e0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:48.710976 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1975c71-282a-4480-afcb-0a5e56a925e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:48.711254 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1975c71-282a-4480-afcb-0a5e56a925e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:48.779060 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1975c71-282a-4480-afcb-0a5e56a925e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:48.781820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1975c71-282a-4480-afcb-0a5e56a925e0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:48.782529 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:48.880361 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 545/1090] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 15:59:48.886242 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96c5ea-36b8-40d1-a385-57f11acf49cc None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:48.989433 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96c5ea-36b8-40d1-a385-57f11acf49cc None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:49.024171 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d195421-025f-4044-8cc2-939aaa550dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:49.025016 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d195421-025f-4044-8cc2-939aaa550dcc None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:49.025293 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d195421-025f-4044-8cc2-939aaa550dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:49.025566 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d195421-025f-4044-8cc2-939aaa550dcc None admin] PATH_INFO: `/v3/projects/5bbb8725157b4a0b8e8f02db1f6a3444` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:49.026910 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d195421-025f-4044-8cc2-939aaa550dcc None admin] RBAC: Authorizing `identity:get_project(project_id=5bbb8725157b4a0b8e8f02db1f6a3444)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:49.028995 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d195421-025f-4044-8cc2-939aaa550dcc None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:49.030737 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 546/1091] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:59:48 2022] GET /identity/v3/projects/5bbb8725157b4a0b8e8f02db1f6a3444 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:49.036812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1975c71-282a-4480-afcb-0a5e56a925e0 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:49.139044 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1975c71-282a-4480-afcb-0a5e56a925e0 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:49.140730 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:49.175704 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8f246-3de2-48ea-97ea-1a7c80a89719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:49.176362 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8f246-3de2-48ea-97ea-1a7c80a89719 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:49.176644 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8f246-3de2-48ea-97ea-1a7c80a89719 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:49.176923 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8f246-3de2-48ea-97ea-1a7c80a89719 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:49.177750 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8f246-3de2-48ea-97ea-1a7c80a89719 None admin] RBAC: Adding query filter params (role.id=bbbc77ec645d4ff5a8bf29ce0377f578, scope.project.id=5bbb8725157b4a0b8e8f02db1f6a3444, user.id=37af54e8e98845179813615798f8d4f7) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:49.178106 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8f246-3de2-48ea-97ea-1a7c80a89719 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:49.179558 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8f246-3de2-48ea-97ea-1a7c80a89719 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:49.186286 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 546/1092] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:59:49 2022] GET /identity/v3/role_assignments?user.id=37af54e8e98845179813615798f8d4f7&scope.project.id=5bbb8725157b4a0b8e8f02db1f6a3444&role.id=bbbc77ec645d4ff5a8bf29ce0377f578 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:49.615550 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbaf79a-62f3-48f3-b5a9-e889434b183a None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:49.615854 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbaf79a-62f3-48f3-b5a9-e889434b183a None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:49.616188 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbaf79a-62f3-48f3-b5a9-e889434b183a None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:49.617569 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 547/1093] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:49.623548 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3365f72-0759-48fd-81f4-a5ecabf1c2a4 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:49.623872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3365f72-0759-48fd-81f4-a5ecabf1c2a4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:49.624210 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3365f72-0759-48fd-81f4-a5ecabf1c2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:49.643389 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d3365f72-0759-48fd-81f4-a5ecabf1c2a4 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:49.644589 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d3365f72-0759-48fd-81f4-a5ecabf1c2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:49.713112 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 547/1094] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:49.737998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbaf79a-62f3-48f3-b5a9-e889434b183a None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:49.811231 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:49.847215 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbaf79a-62f3-48f3-b5a9-e889434b183a None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:49.883112 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3af50d-a602-45a6-8cf5-997b4128123a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:49.883837 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3af50d-a602-45a6-8cf5-997b4128123a None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:49.884372 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3af50d-a602-45a6-8cf5-997b4128123a None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:49.884648 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3af50d-a602-45a6-8cf5-997b4128123a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:49.895181 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3af50d-a602-45a6-8cf5-997b4128123a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:49.904328 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-6f3af50d-a602-45a6-8cf5-997b4128123a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 15:59:49.904328 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:49.904328 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:49.904328 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:49.904328 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:49.904328 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:49.904328 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:49.904328 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:49.904328 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:49.904328 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:49.904328 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:49.904328 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:49.904328 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:49.904328 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:49.904328 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:49.904328 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:49.910348 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:49.910348 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:49.910348 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:49.910348 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:49.910348 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:49.910348 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:49.910348 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:49.910348 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:49.910348 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:49.910348 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:49.910348 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:49.910348 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:49.910348 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:49.910348 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:49.910348 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:49.910348 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:49.911309 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:49.911309 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:49.911309 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:49.911309 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:49.911309 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:49.911309 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:49.911309 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:49.911309 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:49.911309 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:49.911309 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:49.911309 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 15:59:49.911309 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:49.911309 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 548/1095] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jun 16 15:59:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:49.912038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3365f72-0759-48fd-81f4-a5ecabf1c2a4 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:50.009829 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:50.017483 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3365f72-0759-48fd-81f4-a5ecabf1c2a4 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:50.052646 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c5478-911b-4216-88d0-2ad6ed490138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:50.053306 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c5478-911b-4216-88d0-2ad6ed490138 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:50.053586 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c5478-911b-4216-88d0-2ad6ed490138 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:50.053859 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c5478-911b-4216-88d0-2ad6ed490138 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:50.054622 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c5478-911b-4216-88d0-2ad6ed490138 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:50.054925 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c5478-911b-4216-88d0-2ad6ed490138 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:50.056278 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c5478-911b-4216-88d0-2ad6ed490138 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:50.065099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 548/1096] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jun 16 15:59:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:50.071157 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3af50d-a602-45a6-8cf5-997b4128123a None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:50.173828 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3af50d-a602-45a6-8cf5-997b4128123a None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:50.198851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:50.208294 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df452e-1fd5-4f29-9210-81588586bd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:50.209077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df452e-1fd5-4f29-9210-81588586bd74 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:50.209356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df452e-1fd5-4f29-9210-81588586bd74 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:50.209626 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df452e-1fd5-4f29-9210-81588586bd74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:50.221283 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00df452e-1fd5-4f29-9210-81588586bd74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:50.222768 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00df452e-1fd5-4f29-9210-81588586bd74 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:50.239848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-00df452e-1fd5-4f29-9210-81588586bd74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:50.239848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:50.239848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:50.239848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:50.239848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:50.239848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:50.239848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:50.239848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:50.239848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:50.239848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:50.239848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:50.239848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:50.239848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:50.239848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:50.239848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:50.240742 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:50.240742 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:50.240742 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:50.240742 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:50.240742 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:50.240742 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:50.240742 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:50.240742 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:50.240742 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:50.240742 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:50.240742 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:50.240742 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:50.240742 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:50.240742 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:50.240742 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:50.240742 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:50.240742 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:50.241641 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:50.241641 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:50.241641 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:50.241641 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:50.241641 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 549/1097] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:50.247423 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c5478-911b-4216-88d0-2ad6ed490138 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:50.354255 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c5478-911b-4216-88d0-2ad6ed490138 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:50.390500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b158d8d-a006-4ff7-92fb-28d241263a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:50.391169 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b158d8d-a006-4ff7-92fb-28d241263a3d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:50.391451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b158d8d-a006-4ff7-92fb-28d241263a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:50.391724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b158d8d-a006-4ff7-92fb-28d241263a3d None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:50.392442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b158d8d-a006-4ff7-92fb-28d241263a3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:50.392745 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b158d8d-a006-4ff7-92fb-28d241263a3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:50.394132 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b158d8d-a006-4ff7-92fb-28d241263a3d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:50.406096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 549/1098] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:50.411626 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df452e-1fd5-4f29-9210-81588586bd74 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:50.513967 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df452e-1fd5-4f29-9210-81588586bd74 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:50.555792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd238841-6912-44d0-af5c-d9d30d36e2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:50.556603 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd238841-6912-44d0-af5c-d9d30d36e2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:50.556864 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd238841-6912-44d0-af5c-d9d30d36e2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:50.557174 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd238841-6912-44d0-af5c-d9d30d36e2d0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:50.564908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd238841-6912-44d0-af5c-d9d30d36e2d0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:50.566453 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd238841-6912-44d0-af5c-d9d30d36e2d0 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:50.580298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-dd238841-6912-44d0-af5c-d9d30d36e2d0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 15:59:50.580298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:50.580298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:50.580298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:50.580298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:50.580298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:50.580298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:50.580298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:50.580298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:50.580298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:50.580298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:50.580298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:50.580298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:50.580298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:50.580298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:50.580298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:50.581409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:50.581409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:50.581409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:50.581409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:50.581409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:50.581409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:50.581409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:50.581409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:50.581409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:50.581409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:50.581409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:50.581409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:50.581409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:50.581409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:50.581409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:50.581409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:50.581409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:50.581409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:50.582931 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:50.582931 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:50.582931 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:50.582931 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:50.582931 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:50.582931 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:50.582931 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:50.582931 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:50.582931 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 15:59:50.582931 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:50.582931 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 550/1099] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jun 16 15:59:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:50.587214 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b158d8d-a006-4ff7-92fb-28d241263a3d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:50.607566 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:50.689153 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b158d8d-a006-4ff7-92fb-28d241263a3d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:50.725552 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e262edcf-7d3b-4eb2-86b5-29724f00c9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:50.726258 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262edcf-7d3b-4eb2-86b5-29724f00c9bc None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:50.726551 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262edcf-7d3b-4eb2-86b5-29724f00c9bc None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:50.726828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262edcf-7d3b-4eb2-86b5-29724f00c9bc None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:50.727664 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e262edcf-7d3b-4eb2-86b5-29724f00c9bc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:50.727976 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e262edcf-7d3b-4eb2-86b5-29724f00c9bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:50.729427 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e262edcf-7d3b-4eb2-86b5-29724f00c9bc None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:50.748680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 550/1100] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jun 16 15:59:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:50.756656 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd238841-6912-44d0-af5c-d9d30d36e2d0 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:50.803020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:50.866250 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd238841-6912-44d0-af5c-d9d30d36e2d0 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:50.901364 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4ca5c-5612-43bc-b5f5-00eafbaa8f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:50.902116 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4ca5c-5612-43bc-b5f5-00eafbaa8f58 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:50.902504 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4ca5c-5612-43bc-b5f5-00eafbaa8f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:50.902838 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4ca5c-5612-43bc-b5f5-00eafbaa8f58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:50.975054 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4ca5c-5612-43bc-b5f5-00eafbaa8f58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:50.976528 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4ca5c-5612-43bc-b5f5-00eafbaa8f58 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:51.016233 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:51.081734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 551/1101] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 15:59:51.087599 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e262edcf-7d3b-4eb2-86b5-29724f00c9bc None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:51.169801 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:51.193300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e262edcf-7d3b-4eb2-86b5-29724f00c9bc None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:51.227568 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852262ed-6fb3-40ae-bfea-fbd026700b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:51.228363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852262ed-6fb3-40ae-bfea-fbd026700b34 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:51.228645 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852262ed-6fb3-40ae-bfea-fbd026700b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:51.228918 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852262ed-6fb3-40ae-bfea-fbd026700b34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:51.238989 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852262ed-6fb3-40ae-bfea-fbd026700b34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:51.240433 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852262ed-6fb3-40ae-bfea-fbd026700b34 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:51.259819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-852262ed-6fb3-40ae-bfea-fbd026700b34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:51.259819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:51.259819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:51.259819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:51.259819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:51.259819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:51.259819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:51.259819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:51.259819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:51.259819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:51.259819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:51.259819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:51.259819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:51.259819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:51.259819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:51.260800 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:51.260800 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:51.260800 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:51.260800 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:51.260800 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:51.260800 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:51.260800 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:51.260800 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:51.260800 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:51.260800 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:51.260800 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:51.260800 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:51.260800 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:51.260800 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:51.260800 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:51.260800 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:51.260800 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:51.261720 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:51.261720 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:51.261720 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:51.261720 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 15:59:51.261720 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 551/1102] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:51.267577 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4ca5c-5612-43bc-b5f5-00eafbaa8f58 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:51.374507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4ca5c-5612-43bc-b5f5-00eafbaa8f58 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:51.394330 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfd979-741f-4d4e-aa22-53b9718ce286 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:51.394620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfd979-741f-4d4e-aa22-53b9718ce286 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:51.394902 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfd979-741f-4d4e-aa22-53b9718ce286 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:51.396294 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 552/1103] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:59:51.401404 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a7b20-2093-42f5-9441-4edb0e009405 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:51.401690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a7b20-2093-42f5-9441-4edb0e009405 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:51.402120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a7b20-2093-42f5-9441-4edb0e009405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:51.410906 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee67b3a-900d-435b-b013-42073f210b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:51.411571 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee67b3a-900d-435b-b013-42073f210b0c None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:51.411854 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee67b3a-900d-435b-b013-42073f210b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:51.412128 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee67b3a-900d-435b-b013-42073f210b0c None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:51.412851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee67b3a-900d-435b-b013-42073f210b0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:51.413159 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee67b3a-900d-435b-b013-42073f210b0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:51.414586 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee67b3a-900d-435b-b013-42073f210b0c None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:51.420918 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-2e9a7b20-2093-42f5-9441-4edb0e009405 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:51.421455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:51.422180 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2e9a7b20-2093-42f5-9441-4edb0e009405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:51.431864 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 552/1104] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:51.432721 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee67b3a-900d-435b-b013-42073f210b0c None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:51.497786 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 553/1105] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 15:59:51.523424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a7b20-2093-42f5-9441-4edb0e009405 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:51.538449 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee67b3a-900d-435b-b013-42073f210b0c None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:51.573255 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81f682-b6cb-4d55-81af-ede85f386417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:51.574070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81f682-b6cb-4d55-81af-ede85f386417 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:51.574351 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81f682-b6cb-4d55-81af-ede85f386417 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:51.574956 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81f682-b6cb-4d55-81af-ede85f386417 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:51.581583 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81f682-b6cb-4d55-81af-ede85f386417 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:51.581827 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:51.583074 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81f682-b6cb-4d55-81af-ede85f386417 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:51.597817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-ed81f682-b6cb-4d55-81af-ede85f386417 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:51.597817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:51.597817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:51.597817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:51.597817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:51.597817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:51.597817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:51.597817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:51.597817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:51.597817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:51.597817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:51.597817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:51.597817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:51.597817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:51.597817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:51.599011 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:51.599011 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:51.599011 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:51.599011 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:51.599011 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:51.599011 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:51.599011 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:51.599011 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:51.599011 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:51.599011 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:51.599011 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:51.599011 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:51.599011 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:51.599011 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:51.599011 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:51.599011 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:51.599011 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:51.604723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:51.604723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:51.604723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:51.604723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:51.604723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:51.604723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:51.604723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:51.604723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 15:59:51.604723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 553/1106] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:51.605234 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81f682-b6cb-4d55-81af-ede85f386417 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:51.625509 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a7b20-2093-42f5-9441-4edb0e009405 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:51.660450 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e5a0b-a6dd-47fb-bd49-6919961e446e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:51.661245 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e5a0b-a6dd-47fb-bd49-6919961e446e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:51.661530 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e5a0b-a6dd-47fb-bd49-6919961e446e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:51.661813 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e5a0b-a6dd-47fb-bd49-6919961e446e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:51.663028 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e5a0b-a6dd-47fb-bd49-6919961e446e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:51.664464 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e5a0b-a6dd-47fb-bd49-6919961e446e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:51.665034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:51.666157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 554/1107] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:51.672411 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e5a0b-a6dd-47fb-bd49-6919961e446e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:51.709375 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81f682-b6cb-4d55-81af-ede85f386417 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:51.744075 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451fd8c5-9c4b-462f-9b27-b83f85db1528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:51.744743 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fd8c5-9c4b-462f-9b27-b83f85db1528 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:51.745026 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fd8c5-9c4b-462f-9b27-b83f85db1528 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:51.745325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fd8c5-9c4b-462f-9b27-b83f85db1528 None admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:51.746142 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451fd8c5-9c4b-462f-9b27-b83f85db1528 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:51.746583 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451fd8c5-9c4b-462f-9b27-b83f85db1528 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:51.748689 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451fd8c5-9c4b-462f-9b27-b83f85db1528 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:51.760460 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 554/1108] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:51.766507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451fd8c5-9c4b-462f-9b27-b83f85db1528 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:51.773128 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e5a0b-a6dd-47fb-bd49-6919961e446e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:51.799808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:51.809795 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fa265a-80bc-4546-9a86-55f652668c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:51.810500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa265a-80bc-4546-9a86-55f652668c1e None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:51.810788 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa265a-80bc-4546-9a86-55f652668c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:51.811067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa265a-80bc-4546-9a86-55f652668c1e None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:51.811775 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fa265a-80bc-4546-9a86-55f652668c1e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:51.813215 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fa265a-80bc-4546-9a86-55f652668c1e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:51.833819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 555/1109] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 15:59:51 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:59:51.878711 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451fd8c5-9c4b-462f-9b27-b83f85db1528 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:51.898472 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:51.919671 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa362f-a02d-436d-a3ee-271434fee93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:51.920347 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa362f-a02d-436d-a3ee-271434fee93f None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:51.920632 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa362f-a02d-436d-a3ee-271434fee93f None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:51.920907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa362f-a02d-436d-a3ee-271434fee93f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:51.921772 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa362f-a02d-436d-a3ee-271434fee93f None admin] RBAC: Adding query filter params (role.id=758069bda1404a3589771a8f43e95187, scope.project.id=b1177a85a0de4276898daf4a24ff069f, user.id=c3645b1a0be34c01ac87c734fcd85ff6) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:51.922129 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa362f-a02d-436d-a3ee-271434fee93f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:51.923640 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa362f-a02d-436d-a3ee-271434fee93f None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:51.930457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 555/1110] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:59:51 2022] GET /identity/v3/role_assignments?user.id=c3645b1a0be34c01ac87c734fcd85ff6&scope.project.id=b1177a85a0de4276898daf4a24ff069f&role.id=758069bda1404a3589771a8f43e95187 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:54.072561 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d09a4-43ff-4a99-bd1b-3be08ae97380 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:54.073089 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d09a4-43ff-4a99-bd1b-3be08ae97380 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:54.073089 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d09a4-43ff-4a99-bd1b-3be08ae97380 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:54.075019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 556/1111] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:54.080480 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de9c41-d30a-4dd3-91fd-e81433df39f3 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:54.080884 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de9c41-d30a-4dd3-91fd-e81433df39f3 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:54.081036 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de9c41-d30a-4dd3-91fd-e81433df39f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:54.098204 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-59de9c41-d30a-4dd3-91fd-e81433df39f3 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:54.099392 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-59de9c41-d30a-4dd3-91fd-e81433df39f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:54.167605 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 556/1112] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:54.191312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d09a4-43ff-4a99-bd1b-3be08ae97380 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:54.299980 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d09a4-43ff-4a99-bd1b-3be08ae97380 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:54.335015 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c85d0a4-c8c3-4034-be8e-ac2969c16f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:54.335795 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85d0a4-c8c3-4034-be8e-ac2969c16f4c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:54.336074 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85d0a4-c8c3-4034-be8e-ac2969c16f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:54.336344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85d0a4-c8c3-4034-be8e-ac2969c16f4c None admin] PATH_INFO: `/v3/roles/e0b0c216ef51441ab34b6e036da09daa` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:54.337814 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c85d0a4-c8c3-4034-be8e-ac2969c16f4c None admin] RBAC: Authorizing `identity:get_role(role_id=e0b0c216ef51441ab34b6e036da09daa)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:54.339239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c85d0a4-c8c3-4034-be8e-ac2969c16f4c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:54.340688 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 557/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:59:54 2022] GET /identity/v3/roles/e0b0c216ef51441ab34b6e036da09daa => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:54.346582 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de9c41-d30a-4dd3-91fd-e81433df39f3 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:54.444756 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de9c41-d30a-4dd3-91fd-e81433df39f3 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:54.460507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:54.482068 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347beff-2b12-48e7-8a93-65f55a7fb52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:54.482930 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347beff-2b12-48e7-8a93-65f55a7fb52b None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:54.483214 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347beff-2b12-48e7-8a93-65f55a7fb52b None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:54.483491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347beff-2b12-48e7-8a93-65f55a7fb52b None admin] PATH_INFO: `/v3/users/31b225648a7c4f0999b615662fd7acce` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:54.497032 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347beff-2b12-48e7-8a93-65f55a7fb52b None admin] RBAC: Authorizing `identity:get_user(user_id=31b225648a7c4f0999b615662fd7acce)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:54.498674 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347beff-2b12-48e7-8a93-65f55a7fb52b None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:54.500360 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 557/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:59:54 2022] GET /identity/v3/users/31b225648a7c4f0999b615662fd7acce => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:54.506314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c85d0a4-c8c3-4034-be8e-ac2969c16f4c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:54.558485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:54.614242 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c85d0a4-c8c3-4034-be8e-ac2969c16f4c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:54.650236 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f525bf9-6950-444e-b0a7-b2972d8f6819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:54.650893 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f525bf9-6950-444e-b0a7-b2972d8f6819 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:54.651180 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f525bf9-6950-444e-b0a7-b2972d8f6819 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:54.651465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f525bf9-6950-444e-b0a7-b2972d8f6819 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:54.719414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f525bf9-6950-444e-b0a7-b2972d8f6819 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:54.720862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f525bf9-6950-444e-b0a7-b2972d8f6819 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:54.768930 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:54.820556 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 558/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 15:59:54.826721 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347beff-2b12-48e7-8a93-65f55a7fb52b None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:54.927337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347beff-2b12-48e7-8a93-65f55a7fb52b None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:54.962374 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d016f10d-4c1d-4142-a38b-2415de4bd8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:54.963225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016f10d-4c1d-4142-a38b-2415de4bd8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:54.963512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016f10d-4c1d-4142-a38b-2415de4bd8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:54.963792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016f10d-4c1d-4142-a38b-2415de4bd8e1 None admin] PATH_INFO: `/v3/projects/5bbb8725157b4a0b8e8f02db1f6a3444` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:54.964986 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d016f10d-4c1d-4142-a38b-2415de4bd8e1 None admin] RBAC: Authorizing `identity:get_project(project_id=5bbb8725157b4a0b8e8f02db1f6a3444)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:54.966478 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d016f10d-4c1d-4142-a38b-2415de4bd8e1 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:54.968208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 558/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:59:54 2022] GET /identity/v3/projects/5bbb8725157b4a0b8e8f02db1f6a3444 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:54.974333 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f525bf9-6950-444e-b0a7-b2972d8f6819 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:55.073381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f525bf9-6950-444e-b0a7-b2972d8f6819 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:55.111792 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5daa55-4e57-4ac3-a23b-3f9cf5aa8a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:55.112606 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5daa55-4e57-4ac3-a23b-3f9cf5aa8a35 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:55.112993 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5daa55-4e57-4ac3-a23b-3f9cf5aa8a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:55.113388 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5daa55-4e57-4ac3-a23b-3f9cf5aa8a35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:55.114351 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5daa55-4e57-4ac3-a23b-3f9cf5aa8a35 None admin] RBAC: Adding query filter params (role.id=e0b0c216ef51441ab34b6e036da09daa, scope.project.id=5bbb8725157b4a0b8e8f02db1f6a3444, user.id=31b225648a7c4f0999b615662fd7acce) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:55.114766 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5daa55-4e57-4ac3-a23b-3f9cf5aa8a35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:55.116294 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5daa55-4e57-4ac3-a23b-3f9cf5aa8a35 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:55.122689 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 559/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 15:59:54 2022] GET /identity/v3/role_assignments?user.id=31b225648a7c4f0999b615662fd7acce&scope.project.id=5bbb8725157b4a0b8e8f02db1f6a3444&role.id=e0b0c216ef51441ab34b6e036da09daa => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:57.230722 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15620b8c-b371-4055-a7c2-9d1579702b44 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:57.231298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15620b8c-b371-4055-a7c2-9d1579702b44 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:57.231298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15620b8c-b371-4055-a7c2-9d1579702b44 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:57.232655 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 559/1118] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 15:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:57.238718 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f5d6b-1588-4e6b-8f8c-5b96e039c13e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:57.239128 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f5d6b-1588-4e6b-8f8c-5b96e039c13e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:57.239281 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f5d6b-1588-4e6b-8f8c-5b96e039c13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:57.255854 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-cb4f5d6b-1588-4e6b-8f8c-5b96e039c13e None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:57.257049 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-cb4f5d6b-1588-4e6b-8f8c-5b96e039c13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:57.266458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:57.324644 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 560/1119] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 15:59:57.347729 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15620b8c-b371-4055-a7c2-9d1579702b44 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:57.388353 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:57.448161 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15620b8c-b371-4055-a7c2-9d1579702b44 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:57.483442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18679a-f8d4-4f63-822e-57b46f736cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:57.484219 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18679a-f8d4-4f63-822e-57b46f736cec None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:57.484508 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18679a-f8d4-4f63-822e-57b46f736cec None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:57.484782 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18679a-f8d4-4f63-822e-57b46f736cec None admin] PATH_INFO: `/v3/roles/e0b0c216ef51441ab34b6e036da09daa` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:57.486289 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce18679a-f8d4-4f63-822e-57b46f736cec None admin] RBAC: Authorizing `identity:get_role(role_id=e0b0c216ef51441ab34b6e036da09daa)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:57.487669 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce18679a-f8d4-4f63-822e-57b46f736cec None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:57.489062 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 560/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:59:57 2022] GET /identity/v3/roles/e0b0c216ef51441ab34b6e036da09daa => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:57.495037 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f5d6b-1588-4e6b-8f8c-5b96e039c13e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:57.595739 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f5d6b-1588-4e6b-8f8c-5b96e039c13e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:57.629209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354a632-55b5-42cd-9bed-67697dd6dbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:57.630121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354a632-55b5-42cd-9bed-67697dd6dbed None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:57.630404 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354a632-55b5-42cd-9bed-67697dd6dbed None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:57.630680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354a632-55b5-42cd-9bed-67697dd6dbed None admin] PATH_INFO: `/v3/users/31b225648a7c4f0999b615662fd7acce` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:57.631866 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354a632-55b5-42cd-9bed-67697dd6dbed None admin] RBAC: Authorizing `identity:get_user(user_id=31b225648a7c4f0999b615662fd7acce)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:57.633301 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354a632-55b5-42cd-9bed-67697dd6dbed None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:57.635234 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 561/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 15:59:57 2022] GET /identity/v3/users/31b225648a7c4f0999b615662fd7acce => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:57.641883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18679a-f8d4-4f63-822e-57b46f736cec None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:57.744802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18679a-f8d4-4f63-822e-57b46f736cec None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:57.752335 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:57.780467 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597d180-4642-42bc-8d29-474bf1e626c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:57.781331 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597d180-4642-42bc-8d29-474bf1e626c3 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:57.781617 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597d180-4642-42bc-8d29-474bf1e626c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:57.781896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597d180-4642-42bc-8d29-474bf1e626c3 None admin] PATH_INFO: `/v3/projects/5bbb8725157b4a0b8e8f02db1f6a3444` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:57.783153 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597d180-4642-42bc-8d29-474bf1e626c3 None admin] RBAC: Authorizing `identity:get_project(project_id=5bbb8725157b4a0b8e8f02db1f6a3444)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:57.784605 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597d180-4642-42bc-8d29-474bf1e626c3 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:57.786367 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 561/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 15:59:57 2022] GET /identity/v3/projects/5bbb8725157b4a0b8e8f02db1f6a3444 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:57.792275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354a632-55b5-42cd-9bed-67697dd6dbed None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:57.807090 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:57.893746 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354a632-55b5-42cd-9bed-67697dd6dbed None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:57.927472 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebbfe0-9078-44b4-ae03-2ae9b66d5c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:57.928347 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebbfe0-9078-44b4-ae03-2ae9b66d5c66 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:57.928620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebbfe0-9078-44b4-ae03-2ae9b66d5c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:57.928885 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebbfe0-9078-44b4-ae03-2ae9b66d5c66 None admin] PATH_INFO: `/v3/projects/5bbb8725157b4a0b8e8f02db1f6a3444/users/31b225648a7c4f0999b615662fd7acce/roles/e0b0c216ef51441ab34b6e036da09daa` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:57.931067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebbfe0-9078-44b4-ae03-2ae9b66d5c66 None admin] RBAC: Authorizing `identity:create_grant(project_id=5bbb8725157b4a0b8e8f02db1f6a3444, user_id=31b225648a7c4f0999b615662fd7acce, role_id=e0b0c216ef51441ab34b6e036da09daa)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:57.932513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebbfe0-9078-44b4-ae03-2ae9b66d5c66 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:57.946017 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 562/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 15:59:57 2022] PUT /identity/v3/projects/5bbb8725157b4a0b8e8f02db1f6a3444/users/31b225648a7c4f0999b615662fd7acce/roles/e0b0c216ef51441ab34b6e036da09daa => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:00:00.091982 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11dd37-f075-4467-afcb-87151666626e None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:00.092510 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11dd37-f075-4467-afcb-87151666626e None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:00.092633 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11dd37-f075-4467-afcb-87151666626e None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:00.093948 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 562/1124] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:00:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:00.099292 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4e1eb-777a-499c-a9c4-c8bdf22a08c4 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:00.099702 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4e1eb-777a-499c-a9c4-c8bdf22a08c4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:00.099856 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4e1eb-777a-499c-a9c4-c8bdf22a08c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:00.116429 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-94b4e1eb-777a-499c-a9c4-c8bdf22a08c4 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:00.117621 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-94b4e1eb-777a-499c-a9c4-c8bdf22a08c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:00.186608 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 563/1125] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:00.211993 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11dd37-f075-4467-afcb-87151666626e None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:00.327109 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11dd37-f075-4467-afcb-87151666626e None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:00.361438 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172eebf2-d434-48ce-b3ca-bea533079b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:00.362253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172eebf2-d434-48ce-b3ca-bea533079b02 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:00.362539 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172eebf2-d434-48ce-b3ca-bea533079b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:00.362817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172eebf2-d434-48ce-b3ca-bea533079b02 None admin] PATH_INFO: `/v3/roles/e0b0c216ef51441ab34b6e036da09daa` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:00.364305 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172eebf2-d434-48ce-b3ca-bea533079b02 None admin] RBAC: Authorizing `identity:get_role(role_id=e0b0c216ef51441ab34b6e036da09daa)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:00.365777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172eebf2-d434-48ce-b3ca-bea533079b02 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:00.367208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 563/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 16:00:00 2022] GET /identity/v3/roles/e0b0c216ef51441ab34b6e036da09daa => generated 223 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:00.373533 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4e1eb-777a-499c-a9c4-c8bdf22a08c4 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:00.375735 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:00.471748 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4e1eb-777a-499c-a9c4-c8bdf22a08c4 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:00.504345 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7aad0-3a2e-40c4-88fb-0c76099711bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:00.505201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7aad0-3a2e-40c4-88fb-0c76099711bb None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:00.505477 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7aad0-3a2e-40c4-88fb-0c76099711bb None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:00.505747 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7aad0-3a2e-40c4-88fb-0c76099711bb None admin] PATH_INFO: `/v3/users/31b225648a7c4f0999b615662fd7acce` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:00.506941 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a7aad0-3a2e-40c4-88fb-0c76099711bb None admin] RBAC: Authorizing `identity:get_user(user_id=31b225648a7c4f0999b615662fd7acce)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:00.508367 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a7aad0-3a2e-40c4-88fb-0c76099711bb None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:00.510279 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 564/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 16:00:00 2022] GET /identity/v3/users/31b225648a7c4f0999b615662fd7acce => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:00.516641 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172eebf2-d434-48ce-b3ca-bea533079b02 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:00.560454 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:00.619240 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172eebf2-d434-48ce-b3ca-bea533079b02 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:00.653282 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094011e-711a-4df1-9e0c-a4cc88224101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:00.653951 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094011e-711a-4df1-9e0c-a4cc88224101 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:00.654251 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094011e-711a-4df1-9e0c-a4cc88224101 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:00.654512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094011e-711a-4df1-9e0c-a4cc88224101 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:00.728717 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e094011e-711a-4df1-9e0c-a4cc88224101 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:00.730208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e094011e-711a-4df1-9e0c-a4cc88224101 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:00.771794 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:00.832280 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 564/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 16:00:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:00:00.838149 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7aad0-3a2e-40c4-88fb-0c76099711bb None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:00.911183 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:00.943598 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7aad0-3a2e-40c4-88fb-0c76099711bb None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:00.977116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c5796-128e-48da-8ec8-4453fa5868b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:00.977985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c5796-128e-48da-8ec8-4453fa5868b8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:00.978267 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c5796-128e-48da-8ec8-4453fa5868b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:00.978538 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c5796-128e-48da-8ec8-4453fa5868b8 None admin] PATH_INFO: `/v3/projects/5bbb8725157b4a0b8e8f02db1f6a3444` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:00.979698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c5796-128e-48da-8ec8-4453fa5868b8 None admin] RBAC: Authorizing `identity:get_project(project_id=5bbb8725157b4a0b8e8f02db1f6a3444)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:00.981711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c5796-128e-48da-8ec8-4453fa5868b8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:00.983422 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 565/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 16:00:00 2022] GET /identity/v3/projects/5bbb8725157b4a0b8e8f02db1f6a3444 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:00.989523 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094011e-711a-4df1-9e0c-a4cc88224101 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:01.086099 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094011e-711a-4df1-9e0c-a4cc88224101 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:01.121141 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d1e3a7-2a49-430b-825d-6fc157e114e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:01.121880 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1e3a7-2a49-430b-825d-6fc157e114e0 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:01.122188 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1e3a7-2a49-430b-825d-6fc157e114e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:01.122462 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1e3a7-2a49-430b-825d-6fc157e114e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:01.123277 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d1e3a7-2a49-430b-825d-6fc157e114e0 None admin] RBAC: Adding query filter params (role.id=e0b0c216ef51441ab34b6e036da09daa, scope.project.id=5bbb8725157b4a0b8e8f02db1f6a3444, user.id=31b225648a7c4f0999b615662fd7acce) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:01.123585 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d1e3a7-2a49-430b-825d-6fc157e114e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:01.125019 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d1e3a7-2a49-430b-825d-6fc157e114e0 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:01.130825 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 565/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 16:00:00 2022] GET /identity/v3/role_assignments?user.id=31b225648a7c4f0999b615662fd7acce&scope.project.id=5bbb8725157b4a0b8e8f02db1f6a3444&role.id=e0b0c216ef51441ab34b6e036da09daa => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:03.282019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d47d8-5294-4183-a562-2aa65e092347 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:03.282587 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d47d8-5294-4183-a562-2aa65e092347 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:03.282587 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d47d8-5294-4183-a562-2aa65e092347 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:03.283933 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 566/1131] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:03.289188 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d747c5-9c2d-47ca-8aad-dba0560c6d62 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:03.289586 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d747c5-9c2d-47ca-8aad-dba0560c6d62 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:03.289750 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d747c5-9c2d-47ca-8aad-dba0560c6d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:03.301547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:03.307945 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-d4d747c5-9c2d-47ca-8aad-dba0560c6d62 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:03.309675 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-d4d747c5-9c2d-47ca-8aad-dba0560c6d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:03.376919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 566/1132] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:03.400488 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d47d8-5294-4183-a562-2aa65e092347 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:03.504336 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d47d8-5294-4183-a562-2aa65e092347 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:03.539921 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2add295-5b87-4182-907c-ea37811597f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:03.540630 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2add295-5b87-4182-907c-ea37811597f3 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:03.540909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2add295-5b87-4182-907c-ea37811597f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:03.541180 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2add295-5b87-4182-907c-ea37811597f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:03.542355 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2add295-5b87-4182-907c-ea37811597f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:03.544275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2add295-5b87-4182-907c-ea37811597f3 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:03.545887 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 567/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:03.551757 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d747c5-9c2d-47ca-8aad-dba0560c6d62 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:03.653903 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d747c5-9c2d-47ca-8aad-dba0560c6d62 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:03.687569 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d80d7-899f-4d20-9145-4e6f5b4104e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:03.688228 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d80d7-899f-4d20-9145-4e6f5b4104e5 None admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:03.688507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d80d7-899f-4d20-9145-4e6f5b4104e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:03.688783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d80d7-899f-4d20-9145-4e6f5b4104e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:03.689488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d80d7-899f-4d20-9145-4e6f5b4104e5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:03.690960 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d80d7-899f-4d20-9145-4e6f5b4104e5 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:03.695119 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:03.717038 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 567/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jun 16 16:00:03 2022] POST /identity/v3/projects => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:00:05.858825 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184ea3a-b022-448b-ad7e-1327ea7dbe56 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:05.859349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184ea3a-b022-448b-ad7e-1327ea7dbe56 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:05.859459 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184ea3a-b022-448b-ad7e-1327ea7dbe56 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:05.860746 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 568/1135] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:05.865959 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6acf1-1c83-4536-8af6-786705e0de31 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:05.866366 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6acf1-1c83-4536-8af6-786705e0de31 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:05.866549 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6acf1-1c83-4536-8af6-786705e0de31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:05.884269 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-c6f6acf1-1c83-4536-8af6-786705e0de31 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:05.885496 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-c6f6acf1-1c83-4536-8af6-786705e0de31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:05.953980 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 568/1136] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:05.979820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184ea3a-b022-448b-ad7e-1327ea7dbe56 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:05.989741 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:06.082404 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184ea3a-b022-448b-ad7e-1327ea7dbe56 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:06.115566 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64bf4c-4320-48e3-87ce-36663a6967e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:06.116352 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64bf4c-4320-48e3-87ce-36663a6967e9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:06.116636 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64bf4c-4320-48e3-87ce-36663a6967e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:06.116911 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64bf4c-4320-48e3-87ce-36663a6967e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:06.118106 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64bf4c-4320-48e3-87ce-36663a6967e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:06.119537 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64bf4c-4320-48e3-87ce-36663a6967e9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:06.121858 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 569/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:06.127584 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6acf1-1c83-4536-8af6-786705e0de31 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:06.227443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6acf1-1c83-4536-8af6-786705e0de31 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:06.246807 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:06.265662 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f0dd8-bb22-4e93-a7bb-0fb750dfd816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:06.266366 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f0dd8-bb22-4e93-a7bb-0fb750dfd816 None admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:06.266654 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f0dd8-bb22-4e93-a7bb-0fb750dfd816 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:06.266932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f0dd8-bb22-4e93-a7bb-0fb750dfd816 None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:06.267654 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1f0dd8-bb22-4e93-a7bb-0fb750dfd816 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:06.269098 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1f0dd8-bb22-4e93-a7bb-0fb750dfd816 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:06.289432 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 569/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 16:00:06 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:00:08.490545 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7f0e2-ce70-479b-9996-949237811e14 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:08.491076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7f0e2-ce70-479b-9996-949237811e14 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:08.491170 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7f0e2-ce70-479b-9996-949237811e14 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:08.492493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 570/1139] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:08.498272 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d8154-0ad2-405b-a023-08ac74833ded None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:08.498715 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d8154-0ad2-405b-a023-08ac74833ded None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:08.498834 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d8154-0ad2-405b-a023-08ac74833ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:08.516053 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-cf0d8154-0ad2-405b-a023-08ac74833ded None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:08.517266 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-cf0d8154-0ad2-405b-a023-08ac74833ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:08.586853 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 570/1140] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:08.610772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c7f0e2-ce70-479b-9996-949237811e14 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:08.688521 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:08.715497 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c7f0e2-ce70-479b-9996-949237811e14 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:08.750363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab835c9-883d-4e66-8e96-805f812bfddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:08.751075 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab835c9-883d-4e66-8e96-805f812bfddd None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:08.751354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab835c9-883d-4e66-8e96-805f812bfddd None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:08.751626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab835c9-883d-4e66-8e96-805f812bfddd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:08.762922 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab835c9-883d-4e66-8e96-805f812bfddd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:00:08.770357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-8ab835c9-883d-4e66-8e96-805f812bfddd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:00:08.770357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:08.770357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:08.770357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:08.770357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:08.770357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:08.770357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:08.770357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:08.770357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:08.770357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:08.770357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:08.770357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:08.770357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:00:08.770357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:00:08.770357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:00:08.770357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:00:08.771481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:00:08.771481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:00:08.771481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:00:08.771481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:00:08.771481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:08.771481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:08.771481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:08.771481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:08.771481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:08.771481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:08.771481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:08.771481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:08.771481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:08.771481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:08.771481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:08.771481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:08.772370 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:08.772370 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:08.772370 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:08.772370 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:08.772370 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:00:08.772370 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:00:08.772370 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:00:08.772370 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:00:08.772370 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:00:08.772370 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:00:08.772370 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:00:08.772370 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 16:00:08.772370 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 571/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 16:00:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:08.778738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d8154-0ad2-405b-a023-08ac74833ded None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:08.876545 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d8154-0ad2-405b-a023-08ac74833ded None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:08.878852 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:08.912883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13736d39-52ee-468c-beed-ff520544bf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:08.913552 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13736d39-52ee-468c-beed-ff520544bf8d None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:08.913835 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13736d39-52ee-468c-beed-ff520544bf8d None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:08.914130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13736d39-52ee-468c-beed-ff520544bf8d None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:08.914868 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13736d39-52ee-468c-beed-ff520544bf8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:08.915171 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13736d39-52ee-468c-beed-ff520544bf8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:08.916541 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13736d39-52ee-468c-beed-ff520544bf8d None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:08.924882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 571/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 16:00:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:08.931161 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab835c9-883d-4e66-8e96-805f812bfddd None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:09.037956 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab835c9-883d-4e66-8e96-805f812bfddd None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:09.071678 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c717c6-f07f-4779-90b9-77fd056b641a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:09.072544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c717c6-f07f-4779-90b9-77fd056b641a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:09.072831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c717c6-f07f-4779-90b9-77fd056b641a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:09.073127 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c717c6-f07f-4779-90b9-77fd056b641a None admin] PATH_INFO: `/v3/users/95f057c40ca44d80be131588a8aed5ab` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:09.075603 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:09.088390 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c717c6-f07f-4779-90b9-77fd056b641a None admin] RBAC: Authorizing `identity:get_user(user_id=95f057c40ca44d80be131588a8aed5ab)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:09.090501 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c717c6-f07f-4779-90b9-77fd056b641a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:09.092224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 572/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 16:00:08 2022] GET /identity/v3/users/95f057c40ca44d80be131588a8aed5ab => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:09.098412 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13736d39-52ee-468c-beed-ff520544bf8d None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:09.197568 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13736d39-52ee-468c-beed-ff520544bf8d None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:09.231726 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124caaa-b518-41f9-ba9e-459e9d3891fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:09.232369 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124caaa-b518-41f9-ba9e-459e9d3891fc None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:09.232654 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124caaa-b518-41f9-ba9e-459e9d3891fc None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:09.232930 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124caaa-b518-41f9-ba9e-459e9d3891fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:09.266948 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:09.306161 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1124caaa-b518-41f9-ba9e-459e9d3891fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:09.307645 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1124caaa-b518-41f9-ba9e-459e9d3891fc None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:09.407071 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 572/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 16:00:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:00:09.412985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c717c6-f07f-4779-90b9-77fd056b641a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:09.516578 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c717c6-f07f-4779-90b9-77fd056b641a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:09.558649 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c7432-0500-4b55-8ece-9d7527ffde47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:09.560262 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c7432-0500-4b55-8ece-9d7527ffde47 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:09.560556 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c7432-0500-4b55-8ece-9d7527ffde47 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:09.560840 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c7432-0500-4b55-8ece-9d7527ffde47 None admin] PATH_INFO: `/v3/projects/43618d28ec6d40838ef34a49cc853cb7` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:09.562135 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32c7432-0500-4b55-8ece-9d7527ffde47 None admin] RBAC: Authorizing `identity:get_project(project_id=43618d28ec6d40838ef34a49cc853cb7)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:09.563593 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32c7432-0500-4b55-8ece-9d7527ffde47 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:09.565330 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 573/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 16:00:09 2022] GET /identity/v3/projects/43618d28ec6d40838ef34a49cc853cb7 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:09.571599 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124caaa-b518-41f9-ba9e-459e9d3891fc None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:09.638202 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:09.674396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124caaa-b518-41f9-ba9e-459e9d3891fc None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:09.710286 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35366d5-ad02-4c02-b93a-24569520be20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:09.710944 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35366d5-ad02-4c02-b93a-24569520be20 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:09.711395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35366d5-ad02-4c02-b93a-24569520be20 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:09.711627 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35366d5-ad02-4c02-b93a-24569520be20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:09.712878 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35366d5-ad02-4c02-b93a-24569520be20 None admin] RBAC: Adding query filter params (role.id=bbbc77ec645d4ff5a8bf29ce0377f578, scope.project.id=43618d28ec6d40838ef34a49cc853cb7, user.id=95f057c40ca44d80be131588a8aed5ab) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:09.713199 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35366d5-ad02-4c02-b93a-24569520be20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:09.714812 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35366d5-ad02-4c02-b93a-24569520be20 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:09.720514 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 573/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 16:00:09 2022] GET /identity/v3/role_assignments?user.id=95f057c40ca44d80be131588a8aed5ab&scope.project.id=43618d28ec6d40838ef34a49cc853cb7&role.id=bbbc77ec645d4ff5a8bf29ce0377f578 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:11.851096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb784648-bfed-441c-8996-9e1b17a3cb94 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:11.851690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb784648-bfed-441c-8996-9e1b17a3cb94 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:11.851690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb784648-bfed-441c-8996-9e1b17a3cb94 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:11.853009 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 574/1147] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:11.859285 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ce064-8629-418f-872c-6920010accc2 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:11.859754 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ce064-8629-418f-872c-6920010accc2 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:11.859854 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ce064-8629-418f-872c-6920010accc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:11.877307 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-166ce064-8629-418f-872c-6920010accc2 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:11.878575 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-166ce064-8629-418f-872c-6920010accc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:11.950918 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 574/1148] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:11.974070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb784648-bfed-441c-8996-9e1b17a3cb94 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:12.036000 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:12.079816 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb784648-bfed-441c-8996-9e1b17a3cb94 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:12.115715 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab39c63-3ca3-44a6-a367-4b8cf5aed158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:12.116501 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab39c63-3ca3-44a6-a367-4b8cf5aed158 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:12.116775 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab39c63-3ca3-44a6-a367-4b8cf5aed158 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:12.117047 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab39c63-3ca3-44a6-a367-4b8cf5aed158 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:12.127826 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab39c63-3ca3-44a6-a367-4b8cf5aed158 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:00:12.135577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-3ab39c63-3ca3-44a6-a367-4b8cf5aed158 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:00:12.135577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:12.135577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:12.135577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:12.135577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:12.135577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:12.135577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:12.135577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:12.135577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:12.135577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:12.135577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:12.135577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:12.135577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:00:12.135577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:00:12.135577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:00:12.135577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:00:12.136760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:00:12.136760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:00:12.136760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:00:12.136760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:00:12.136760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:12.136760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:12.136760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:12.136760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:12.136760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:12.136760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:12.136760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:12.136760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:12.136760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:12.136760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:12.136760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:12.136760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:12.137752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:12.137752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:12.137752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:12.137752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:12.137752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:00:12.137752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:00:12.137752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:00:12.137752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:00:12.137752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:00:12.137752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:00:12.137752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:00:12.137752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 16:00:12.137752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 575/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 16:00:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:12.143176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ce064-8629-418f-872c-6920010accc2 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:12.201466 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:12.243398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ce064-8629-418f-872c-6920010accc2 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:12.277544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c446b198-0481-4a3e-9960-98b72c614b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:12.278228 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446b198-0481-4a3e-9960-98b72c614b5d None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:12.278510 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446b198-0481-4a3e-9960-98b72c614b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:12.278782 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446b198-0481-4a3e-9960-98b72c614b5d None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:12.279516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c446b198-0481-4a3e-9960-98b72c614b5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:12.279819 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c446b198-0481-4a3e-9960-98b72c614b5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:12.281184 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c446b198-0481-4a3e-9960-98b72c614b5d None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:12.289089 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 575/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 16:00:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:12.295954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab39c63-3ca3-44a6-a367-4b8cf5aed158 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:12.399837 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab39c63-3ca3-44a6-a367-4b8cf5aed158 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:12.421660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:12.440032 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d014d1-f092-4ffe-92b2-0f25f416f2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:12.440915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d014d1-f092-4ffe-92b2-0f25f416f2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:12.441198 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d014d1-f092-4ffe-92b2-0f25f416f2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:12.441475 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d014d1-f092-4ffe-92b2-0f25f416f2c1 None admin] PATH_INFO: `/v3/users/95f057c40ca44d80be131588a8aed5ab` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:12.442701 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d014d1-f092-4ffe-92b2-0f25f416f2c1 None admin] RBAC: Authorizing `identity:get_user(user_id=95f057c40ca44d80be131588a8aed5ab)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:12.444141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d014d1-f092-4ffe-92b2-0f25f416f2c1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:12.445780 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 576/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 16:00:12 2022] GET /identity/v3/users/95f057c40ca44d80be131588a8aed5ab => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:12.451720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c446b198-0481-4a3e-9960-98b72c614b5d None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:12.552884 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c446b198-0481-4a3e-9960-98b72c614b5d None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:12.575823 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:12.588547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f7551-a7ce-4757-9e09-fbe449377e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:12.590211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f7551-a7ce-4757-9e09-fbe449377e5a None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:12.590487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f7551-a7ce-4757-9e09-fbe449377e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:12.590773 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f7551-a7ce-4757-9e09-fbe449377e5a None admin] PATH_INFO: `/v3/projects/43618d28ec6d40838ef34a49cc853cb7` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:12.591989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f7551-a7ce-4757-9e09-fbe449377e5a None admin] RBAC: Authorizing `identity:get_project(project_id=43618d28ec6d40838ef34a49cc853cb7)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:12.593482 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f7551-a7ce-4757-9e09-fbe449377e5a None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:12.595241 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 576/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 16:00:12 2022] GET /identity/v3/projects/43618d28ec6d40838ef34a49cc853cb7 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:12.601150 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d014d1-f092-4ffe-92b2-0f25f416f2c1 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:12.703323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d014d1-f092-4ffe-92b2-0f25f416f2c1 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:12.737058 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ee34d-47d0-41a9-986e-7f12a6b3a662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:12.737967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ee34d-47d0-41a9-986e-7f12a6b3a662 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:12.738253 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ee34d-47d0-41a9-986e-7f12a6b3a662 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:12.738526 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ee34d-47d0-41a9-986e-7f12a6b3a662 None admin] PATH_INFO: `/v3/projects/43618d28ec6d40838ef34a49cc853cb7/users/95f057c40ca44d80be131588a8aed5ab/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:12.740484 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ee34d-47d0-41a9-986e-7f12a6b3a662 None admin] RBAC: Authorizing `identity:create_grant(project_id=43618d28ec6d40838ef34a49cc853cb7, user_id=95f057c40ca44d80be131588a8aed5ab, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:12.741919 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ee34d-47d0-41a9-986e-7f12a6b3a662 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:12.756304 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 577/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 16:00:12 2022] PUT /identity/v3/projects/43618d28ec6d40838ef34a49cc853cb7/users/95f057c40ca44d80be131588a8aed5ab/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:00:14.879365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc89c4-ff2f-4e6b-b451-cec65be663ad None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:14.879914 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc89c4-ff2f-4e6b-b451-cec65be663ad None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:14.879914 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc89c4-ff2f-4e6b-b451-cec65be663ad None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:14.881290 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 577/1154] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:14.887128 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87f409-734c-4556-a09b-635bced13da6 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:14.887128 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87f409-734c-4556-a09b-635bced13da6 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:14.887956 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87f409-734c-4556-a09b-635bced13da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:14.906203 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-da87f409-734c-4556-a09b-635bced13da6 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:14.907433 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-da87f409-734c-4556-a09b-635bced13da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:14.950522 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:14.974837 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 578/1155] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:14.999548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cc89c4-ff2f-4e6b-b451-cec65be663ad None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:15.112396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cc89c4-ff2f-4e6b-b451-cec65be663ad None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:15.149827 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34de0e-424e-4b62-8c06-f6473856a3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:15.150642 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34de0e-424e-4b62-8c06-f6473856a3d4 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:15.150933 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34de0e-424e-4b62-8c06-f6473856a3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:15.151211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34de0e-424e-4b62-8c06-f6473856a3d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:15.165436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed34de0e-424e-4b62-8c06-f6473856a3d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:00:15.173278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-ed34de0e-424e-4b62-8c06-f6473856a3d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:00:15.173278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:15.173278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:15.173278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:15.173278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:15.173278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:15.173278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:15.173278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:15.173278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:15.173278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:15.173278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:15.173278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:15.173278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:00:15.173278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:00:15.173278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:00:15.173278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:00:15.174442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:00:15.174442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:00:15.174442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:00:15.174442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:00:15.174442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:15.174442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:15.174442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:15.174442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:15.174442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:15.174442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:15.174442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:15.174442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:15.174442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:15.174442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:15.174442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:15.174442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:15.175691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:15.175691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:15.175691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:15.175691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:15.175691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:00:15.175691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:00:15.175691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:00:15.175691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:00:15.175691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:00:15.175691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:00:15.175691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:00:15.175691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 16:00:15.175691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 578/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 16:00:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:15.180951 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87f409-734c-4556-a09b-635bced13da6 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:15.285630 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87f409-734c-4556-a09b-635bced13da6 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:15.319527 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810216e5-6e39-42af-b1f6-98cc642fb996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:15.320217 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810216e5-6e39-42af-b1f6-98cc642fb996 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:15.320519 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810216e5-6e39-42af-b1f6-98cc642fb996 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:15.320809 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810216e5-6e39-42af-b1f6-98cc642fb996 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:15.321562 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810216e5-6e39-42af-b1f6-98cc642fb996 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:15.321871 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810216e5-6e39-42af-b1f6-98cc642fb996 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:15.323285 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810216e5-6e39-42af-b1f6-98cc642fb996 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:15.332834 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 579/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 16:00:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:15.340831 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34de0e-424e-4b62-8c06-f6473856a3d4 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:15.393448 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:15.445270 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34de0e-424e-4b62-8c06-f6473856a3d4 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:15.481949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1687b-7445-4eed-8d4a-4ff47033e6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:15.482872 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1687b-7445-4eed-8d4a-4ff47033e6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:15.483153 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1687b-7445-4eed-8d4a-4ff47033e6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:15.483443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1687b-7445-4eed-8d4a-4ff47033e6b0 None admin] PATH_INFO: `/v3/users/95f057c40ca44d80be131588a8aed5ab` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:15.484649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b1687b-7445-4eed-8d4a-4ff47033e6b0 None admin] RBAC: Authorizing `identity:get_user(user_id=95f057c40ca44d80be131588a8aed5ab)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:15.486324 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b1687b-7445-4eed-8d4a-4ff47033e6b0 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:15.487990 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 579/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 16:00:15 2022] GET /identity/v3/users/95f057c40ca44d80be131588a8aed5ab => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:15.493882 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810216e5-6e39-42af-b1f6-98cc642fb996 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:15.538433 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:15.597689 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810216e5-6e39-42af-b1f6-98cc642fb996 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:15.631103 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7ab8e-4bc5-45eb-8992-593b39225e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:15.631748 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7ab8e-4bc5-45eb-8992-593b39225e77 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:15.632034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7ab8e-4bc5-45eb-8992-593b39225e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:15.632311 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7ab8e-4bc5-45eb-8992-593b39225e77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:15.713231 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7ab8e-4bc5-45eb-8992-593b39225e77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:15.714981 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7ab8e-4bc5-45eb-8992-593b39225e77 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:15.760956 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:15.851111 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 580/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 16:00:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:00:15.857201 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1687b-7445-4eed-8d4a-4ff47033e6b0 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:15.969489 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1687b-7445-4eed-8d4a-4ff47033e6b0 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:15.988249 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:16.007988 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af9a1f-bfe2-4cda-8dcb-997f64e61539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:16.009120 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af9a1f-bfe2-4cda-8dcb-997f64e61539 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:16.009397 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af9a1f-bfe2-4cda-8dcb-997f64e61539 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:16.009668 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af9a1f-bfe2-4cda-8dcb-997f64e61539 None admin] PATH_INFO: `/v3/projects/43618d28ec6d40838ef34a49cc853cb7` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:16.011133 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af9a1f-bfe2-4cda-8dcb-997f64e61539 None admin] RBAC: Authorizing `identity:get_project(project_id=43618d28ec6d40838ef34a49cc853cb7)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:16.013628 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af9a1f-bfe2-4cda-8dcb-997f64e61539 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:16.015512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 580/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 16:00:15 2022] GET /identity/v3/projects/43618d28ec6d40838ef34a49cc853cb7 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:16.022475 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7ab8e-4bc5-45eb-8992-593b39225e77 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:16.137801 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7ab8e-4bc5-45eb-8992-593b39225e77 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:16.177697 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a00ae-8435-439a-8012-3e2c3be22818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:16.178395 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a00ae-8435-439a-8012-3e2c3be22818 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:16.178683 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a00ae-8435-439a-8012-3e2c3be22818 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:16.178962 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a00ae-8435-439a-8012-3e2c3be22818 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:16.179866 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1a00ae-8435-439a-8012-3e2c3be22818 None admin] RBAC: Adding query filter params (role.id=bbbc77ec645d4ff5a8bf29ce0377f578, scope.project.id=43618d28ec6d40838ef34a49cc853cb7, user.id=95f057c40ca44d80be131588a8aed5ab) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:16.180181 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1a00ae-8435-439a-8012-3e2c3be22818 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:16.181635 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1a00ae-8435-439a-8012-3e2c3be22818 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:16.191252 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 581/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 16:00:16 2022] GET /identity/v3/role_assignments?user.id=95f057c40ca44d80be131588a8aed5ab&scope.project.id=43618d28ec6d40838ef34a49cc853cb7&role.id=bbbc77ec645d4ff5a8bf29ce0377f578 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:18.379713 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5bba1-5989-4881-88de-62e497c83f8d None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:18.380248 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5bba1-5989-4881-88de-62e497c83f8d None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:18.380248 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5bba1-5989-4881-88de-62e497c83f8d None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:18.381634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 581/1162] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:18.387690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc782e-3fed-4b2d-89a1-7949b5fcc8ec None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:18.388084 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc782e-3fed-4b2d-89a1-7949b5fcc8ec None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:18.388241 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc782e-3fed-4b2d-89a1-7949b5fcc8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:18.394172 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:18.405591 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-43bc782e-3fed-4b2d-89a1-7949b5fcc8ec None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:18.406810 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-43bc782e-3fed-4b2d-89a1-7949b5fcc8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:18.474255 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 582/1163] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:18.501558 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f5bba1-5989-4881-88de-62e497c83f8d None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:18.606646 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f5bba1-5989-4881-88de-62e497c83f8d None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:18.644425 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b0574-d060-42b5-bb96-80486734c182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:18.645007 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b0574-d060-42b5-bb96-80486734c182 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:18.645376 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b0574-d060-42b5-bb96-80486734c182 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:18.645711 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b0574-d060-42b5-bb96-80486734c182 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:18.701381 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:18.715678 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b0574-d060-42b5-bb96-80486734c182 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:18.717150 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b0574-d060-42b5-bb96-80486734c182 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:18.816576 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 582/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 16:00:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:00:18.822659 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc782e-3fed-4b2d-89a1-7949b5fcc8ec None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:18.931083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc782e-3fed-4b2d-89a1-7949b5fcc8ec None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:18.986322 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011df71d-b67c-4a91-9f4b-1bda5ae866c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:18.987133 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011df71d-b67c-4a91-9f4b-1bda5ae866c1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:18.987418 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011df71d-b67c-4a91-9f4b-1bda5ae866c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:18.987695 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011df71d-b67c-4a91-9f4b-1bda5ae866c1 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:18.989700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:18.997028 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011df71d-b67c-4a91-9f4b-1bda5ae866c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:18.998520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011df71d-b67c-4a91-9f4b-1bda5ae866c1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:19.013817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-011df71d-b67c-4a91-9f4b-1bda5ae866c1 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 16:00:19.013817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:19.013817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 16 16:00:19.013817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 16 16:00:19.013817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:00:19.013817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:00:19.013817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:00:19.013817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:00:19.013817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 16 16:00:19.013817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 16:00:19.013817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:00:19.013817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 16:00:19.013817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:19.013817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:19.013817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:19.013817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:19.013817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:19.015774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:19.015774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:19.015774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:19.015774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:19.015774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:19.015774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:19.015774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:19.015774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:19.015774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:19.015774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:19.015774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:19.015774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:19.015774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:19.015774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:19.015774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:19.015774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:19.017381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:19.017381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:19.017381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:19.017381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:19.017381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:19.017381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:19.017381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:19.017381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:19.017381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:19.017381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:19.017381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 16 16:00:19.017381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:19.017381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 16:00:19.017381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 16:00:19.017381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 583/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jun 16 16:00:18 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:19.021742 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b0574-d060-42b5-bb96-80486734c182 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:19.115125 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:19.132437 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b0574-d060-42b5-bb96-80486734c182 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:19.171507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b3810-77f0-4fb0-90fb-7957addafbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:19.172176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b3810-77f0-4fb0-90fb-7957addafbee None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:19.172461 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b3810-77f0-4fb0-90fb-7957addafbee None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:19.172737 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b3810-77f0-4fb0-90fb-7957addafbee None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:19.173468 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b3810-77f0-4fb0-90fb-7957addafbee None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:19.173769 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b3810-77f0-4fb0-90fb-7957addafbee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:19.175199 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b3810-77f0-4fb0-90fb-7957addafbee None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:19.184249 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 583/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jun 16 16:00:19 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:21.385569 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a33f6-a7a3-465a-bcd5-7caf066415b3 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:21.386665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a33f6-a7a3-465a-bcd5-7caf066415b3 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:21.386665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a33f6-a7a3-465a-bcd5-7caf066415b3 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:21.388243 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 584/1167] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:00:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:21.393590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe84cb-0985-4d0f-aa1e-137e93853990 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:21.394054 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe84cb-0985-4d0f-aa1e-137e93853990 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:21.394180 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe84cb-0985-4d0f-aa1e-137e93853990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:21.413260 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-d2fe84cb-0985-4d0f-aa1e-137e93853990 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:21.414492 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-d2fe84cb-0985-4d0f-aa1e-137e93853990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:21.482485 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 584/1168] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:21.506691 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a33f6-a7a3-465a-bcd5-7caf066415b3 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:21.615266 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a33f6-a7a3-465a-bcd5-7caf066415b3 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:21.650868 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b3be06-deb6-46ce-98b4-c650b77fa07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:21.651494 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3be06-deb6-46ce-98b4-c650b77fa07f None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:21.651775 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3be06-deb6-46ce-98b4-c650b77fa07f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:21.652049 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3be06-deb6-46ce-98b4-c650b77fa07f None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:21.652801 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b3be06-deb6-46ce-98b4-c650b77fa07f None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:21.654275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b3be06-deb6-46ce-98b4-c650b77fa07f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:21.669531 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 585/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Thu Jun 16 16:00:21 2022] POST /identity/v3/domains => generated 267 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:00:23.869476 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2b9d9-bc24-420c-8caf-a5acd1b11ab4 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:23.870038 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2b9d9-bc24-420c-8caf-a5acd1b11ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:23.870135 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2b9d9-bc24-420c-8caf-a5acd1b11ab4 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:23.871547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 585/1170] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:23.876953 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d45357-594f-43e0-90a1-6221adda3521 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:23.877362 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d45357-594f-43e0-90a1-6221adda3521 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:23.877521 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d45357-594f-43e0-90a1-6221adda3521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:23.894665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-88d45357-594f-43e0-90a1-6221adda3521 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:23.895853 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-88d45357-594f-43e0-90a1-6221adda3521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:23.944657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:23.962756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 586/1171] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:23.987567 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2b9d9-bc24-420c-8caf-a5acd1b11ab4 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:24.070762 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:24.098975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2b9d9-bc24-420c-8caf-a5acd1b11ab4 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:24.134225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae33ad16-13c2-4343-b56e-55f2a200eec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:24.135107 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33ad16-13c2-4343-b56e-55f2a200eec5 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:24.135395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33ad16-13c2-4343-b56e-55f2a200eec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:24.136296 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33ad16-13c2-4343-b56e-55f2a200eec5 None admin] PATH_INFO: `/v3/domains/62665ea122a24ce4a59d6bd62d6fd33f` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:24.147426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33ad16-13c2-4343-b56e-55f2a200eec5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=62665ea122a24ce4a59d6bd62d6fd33f)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:24.148896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33ad16-13c2-4343-b56e-55f2a200eec5 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:24.150839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 586/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Thu Jun 16 16:00:23 2022] GET /identity/v3/domains/62665ea122a24ce4a59d6bd62d6fd33f => generated 267 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:24.157699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d45357-594f-43e0-90a1-6221adda3521 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:24.253224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d45357-594f-43e0-90a1-6221adda3521 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:24.287879 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c51115-4026-4386-9134-70cb8e67edff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:24.288541 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c51115-4026-4386-9134-70cb8e67edff None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:24.288829 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c51115-4026-4386-9134-70cb8e67edff None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:24.289116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c51115-4026-4386-9134-70cb8e67edff None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:24.289816 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c51115-4026-4386-9134-70cb8e67edff None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:24.291331 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c51115-4026-4386-9134-70cb8e67edff None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:24.318105 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 587/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jun 16 16:00:24 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:00:26.520393 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6307545-9544-42b0-8cb1-cd504e2f3045 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:26.520931 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6307545-9544-42b0-8cb1-cd504e2f3045 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:26.520931 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6307545-9544-42b0-8cb1-cd504e2f3045 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:26.522364 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 587/1174] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:26.528185 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7ce0b-b174-4996-9206-10b37be81d36 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:26.528624 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7ce0b-b174-4996-9206-10b37be81d36 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:26.528732 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7ce0b-b174-4996-9206-10b37be81d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:26.545428 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e5e7ce0b-b174-4996-9206-10b37be81d36 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:26.546643 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e5e7ce0b-b174-4996-9206-10b37be81d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:26.575894 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:26.617139 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 588/1175] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:00:26.642997 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6307545-9544-42b0-8cb1-cd504e2f3045 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:26.744208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6307545-9544-42b0-8cb1-cd504e2f3045 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:26.778500 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1bf5c5-9092-456a-af0b-a2f9c976fbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:26.779301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bf5c5-9092-456a-af0b-a2f9c976fbb8 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:26.779586 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bf5c5-9092-456a-af0b-a2f9c976fbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:26.779873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bf5c5-9092-456a-af0b-a2f9c976fbb8 None admin] PATH_INFO: `/v3/domains/62665ea122a24ce4a59d6bd62d6fd33f` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:26.781087 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1bf5c5-9092-456a-af0b-a2f9c976fbb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=62665ea122a24ce4a59d6bd62d6fd33f)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:26.782596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1bf5c5-9092-456a-af0b-a2f9c976fbb8 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:26.784251 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 588/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Thu Jun 16 16:00:26 2022] GET /identity/v3/domains/62665ea122a24ce4a59d6bd62d6fd33f => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:26.790093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7ce0b-b174-4996-9206-10b37be81d36 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:26.890046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7ce0b-b174-4996-9206-10b37be81d36 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:26.923101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86a15e-d757-4ef8-a170-68a62726fdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:26.923773 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86a15e-d757-4ef8-a170-68a62726fdb6 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:26.924056 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86a15e-d757-4ef8-a170-68a62726fdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:26.924332 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86a15e-d757-4ef8-a170-68a62726fdb6 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:26.925133 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86a15e-d757-4ef8-a170-68a62726fdb6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:26.926596 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86a15e-d757-4ef8-a170-68a62726fdb6 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:26.946552 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 589/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 16:00:26 2022] POST /identity/v3/users => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:00:29.129726 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff88cb-f3b6-4347-b948-67209d234726 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:29.130298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff88cb-f3b6-4347-b948-67209d234726 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:29.130386 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff88cb-f3b6-4347-b948-67209d234726 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:29.131721 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 589/1178] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:29.137286 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4610d24-3191-4640-b6ec-ee3f72a71715 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:29.137706 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4610d24-3191-4640-b6ec-ee3f72a71715 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:29.137850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4610d24-3191-4640-b6ec-ee3f72a71715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:29.155019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d4610d24-3191-4640-b6ec-ee3f72a71715 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:29.156398 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d4610d24-3191-4640-b6ec-ee3f72a71715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:29.174063 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:29.225564 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 590/1179] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:29.249821 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff88cb-f3b6-4347-b948-67209d234726 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:29.261982 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:29.356131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff88cb-f3b6-4347-b948-67209d234726 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:29.390510 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b084b510-2085-448e-8e8b-5f13752995d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:29.391226 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084b510-2085-448e-8e8b-5f13752995d7 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:29.391512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084b510-2085-448e-8e8b-5f13752995d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:29.391781 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084b510-2085-448e-8e8b-5f13752995d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:29.403146 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b084b510-2085-448e-8e8b-5f13752995d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:00:29.410547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-b084b510-2085-448e-8e8b-5f13752995d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:00:29.410547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:29.410547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:29.410547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:29.410547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:29.410547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:29.410547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:29.410547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:29.410547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:29.410547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:29.410547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:29.410547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:29.410547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:00:29.410547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:00:29.410547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:00:29.410547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:00:29.411650 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:00:29.411650 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:00:29.411650 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:00:29.411650 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:00:29.411650 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:29.411650 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:29.411650 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:29.411650 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:29.411650 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:29.411650 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:29.411650 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:29.411650 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:29.411650 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:29.411650 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:29.411650 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:29.411650 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:29.412708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:29.412708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:29.412708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:29.412708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:29.412708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:00:29.412708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:00:29.412708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:00:29.412708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:00:29.412708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:00:29.412708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:00:29.412708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:00:29.412708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 16:00:29.412708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 590/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 16:00:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:29.417867 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4610d24-3191-4640-b6ec-ee3f72a71715 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:29.516335 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4610d24-3191-4640-b6ec-ee3f72a71715 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:29.549784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79525a21-e007-4805-8c6a-429893f1c05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:29.550469 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79525a21-e007-4805-8c6a-429893f1c05d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:29.550754 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79525a21-e007-4805-8c6a-429893f1c05d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:29.551033 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79525a21-e007-4805-8c6a-429893f1c05d None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:29.551772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79525a21-e007-4805-8c6a-429893f1c05d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:29.552085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79525a21-e007-4805-8c6a-429893f1c05d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:29.553448 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79525a21-e007-4805-8c6a-429893f1c05d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:29.561497 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 591/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 16:00:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:29.567412 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b084b510-2085-448e-8e8b-5f13752995d7 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:29.633858 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:29.667716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b084b510-2085-448e-8e8b-5f13752995d7 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:29.700796 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27d5a4-90d1-44af-87a8-53a0324d8078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:29.701660 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27d5a4-90d1-44af-87a8-53a0324d8078 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:29.701954 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27d5a4-90d1-44af-87a8-53a0324d8078 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:29.702236 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27d5a4-90d1-44af-87a8-53a0324d8078 None admin] PATH_INFO: `/v3/users/4e51845ddf1f4063a58d7fd540f4aacf` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:29.716131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27d5a4-90d1-44af-87a8-53a0324d8078 None admin] RBAC: Authorizing `identity:get_user(user_id=4e51845ddf1f4063a58d7fd540f4aacf)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:29.717590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27d5a4-90d1-44af-87a8-53a0324d8078 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:29.719313 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 591/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 16:00:29 2022] GET /identity/v3/users/4e51845ddf1f4063a58d7fd540f4aacf => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:29.725303 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79525a21-e007-4805-8c6a-429893f1c05d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:29.741453 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:29.834032 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79525a21-e007-4805-8c6a-429893f1c05d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:29.868024 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7f3ad-2396-4952-b501-fe1b88f78446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:29.868667 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7f3ad-2396-4952-b501-fe1b88f78446 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:29.868955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7f3ad-2396-4952-b501-fe1b88f78446 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:29.869233 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7f3ad-2396-4952-b501-fe1b88f78446 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:29.936736 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7f3ad-2396-4952-b501-fe1b88f78446 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:29.938157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7f3ad-2396-4952-b501-fe1b88f78446 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:29.952667 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:30.037008 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 592/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 16:00:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:00:30.043625 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27d5a4-90d1-44af-87a8-53a0324d8078 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:30.145727 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27d5a4-90d1-44af-87a8-53a0324d8078 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:30.170976 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:30.179982 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7698e292-f506-4f6f-b995-77181fe44657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:30.180829 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698e292-f506-4f6f-b995-77181fe44657 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:30.181110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698e292-f506-4f6f-b995-77181fe44657 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:30.181383 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698e292-f506-4f6f-b995-77181fe44657 None admin] PATH_INFO: `/v3/projects/c7d1258df72d466da6fe2f7910f0754b` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:30.182650 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7698e292-f506-4f6f-b995-77181fe44657 None admin] RBAC: Authorizing `identity:get_project(project_id=c7d1258df72d466da6fe2f7910f0754b)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:30.184110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7698e292-f506-4f6f-b995-77181fe44657 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:30.185806 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 592/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 16:00:30 2022] GET /identity/v3/projects/c7d1258df72d466da6fe2f7910f0754b => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:30.191975 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7f3ad-2396-4952-b501-fe1b88f78446 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:30.294922 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7f3ad-2396-4952-b501-fe1b88f78446 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:30.309657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:30.331143 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd8851-3572-43fa-babb-48c3f1c1d8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:30.331801 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd8851-3572-43fa-babb-48c3f1c1d8cd None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:30.332085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd8851-3572-43fa-babb-48c3f1c1d8cd None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:30.332366 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd8851-3572-43fa-babb-48c3f1c1d8cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:30.333220 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd8851-3572-43fa-babb-48c3f1c1d8cd None admin] RBAC: Adding query filter params (role.id=bbbc77ec645d4ff5a8bf29ce0377f578, scope.project.id=c7d1258df72d466da6fe2f7910f0754b, user.id=4e51845ddf1f4063a58d7fd540f4aacf) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:30.333537 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd8851-3572-43fa-babb-48c3f1c1d8cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:30.335006 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd8851-3572-43fa-babb-48c3f1c1d8cd None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:30.341363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 593/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 16:00:30 2022] GET /identity/v3/role_assignments?user.id=4e51845ddf1f4063a58d7fd540f4aacf&scope.project.id=c7d1258df72d466da6fe2f7910f0754b&role.id=bbbc77ec645d4ff5a8bf29ce0377f578 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:32.476113 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a46c0-dec4-465b-8493-58914ca9646d None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:32.476669 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a46c0-dec4-465b-8493-58914ca9646d None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:32.477103 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a46c0-dec4-465b-8493-58914ca9646d None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:32.479709 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 593/1186] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:00:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:32.485189 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ede40-267e-4a1e-b655-db45247e5252 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:32.485661 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ede40-267e-4a1e-b655-db45247e5252 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:32.485748 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ede40-267e-4a1e-b655-db45247e5252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:32.503215 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-5f3ede40-267e-4a1e-b655-db45247e5252 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:32.504437 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5f3ede40-267e-4a1e-b655-db45247e5252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:32.576702 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 594/1187] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:32.599736 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a46c0-dec4-465b-8493-58914ca9646d None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:32.700800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a46c0-dec4-465b-8493-58914ca9646d None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:32.734773 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611fb38-07fa-49b7-9671-6bc9a599a480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:32.735485 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611fb38-07fa-49b7-9671-6bc9a599a480 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:32.735771 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611fb38-07fa-49b7-9671-6bc9a599a480 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:32.736044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611fb38-07fa-49b7-9671-6bc9a599a480 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:32.746101 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9611fb38-07fa-49b7-9671-6bc9a599a480 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:00:32.753720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-9611fb38-07fa-49b7-9671-6bc9a599a480 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:00:32.753720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:32.753720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:32.753720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:32.753720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:32.753720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:32.753720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:32.753720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:32.753720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:32.753720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:32.753720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:32.753720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:32.753720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:00:32.753720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:00:32.753720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:00:32.753720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:00:32.754712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:00:32.754712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:00:32.754712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:00:32.754712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:00:32.754712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:32.754712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:32.754712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:32.754712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:32.754712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:32.754712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:32.754712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:32.754712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:32.754712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:32.754712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:32.754712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:32.754712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:32.755570 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:32.755570 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:32.755570 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:32.755570 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:32.755570 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:00:32.755570 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:00:32.755570 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:00:32.755570 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:00:32.755570 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:00:32.755570 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:00:32.755570 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:00:32.755570 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 16:00:32.755570 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 594/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 16:00:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:32.762298 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ede40-267e-4a1e-b655-db45247e5252 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:32.862165 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ede40-267e-4a1e-b655-db45247e5252 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:32.871611 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:32.897751 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e740525-5e82-45f6-bd8d-e35ec6ebafec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:32.898454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e740525-5e82-45f6-bd8d-e35ec6ebafec None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:32.898741 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e740525-5e82-45f6-bd8d-e35ec6ebafec None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:32.899021 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e740525-5e82-45f6-bd8d-e35ec6ebafec None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:32.899812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e740525-5e82-45f6-bd8d-e35ec6ebafec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:32.900078 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e740525-5e82-45f6-bd8d-e35ec6ebafec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:32.901487 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e740525-5e82-45f6-bd8d-e35ec6ebafec None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:32.909590 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 595/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 16:00:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:32.915441 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611fb38-07fa-49b7-9671-6bc9a599a480 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:32.971830 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:33.018251 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611fb38-07fa-49b7-9671-6bc9a599a480 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:33.052543 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319d365-4deb-4a02-8dc7-76ed3dfc432f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:33.053411 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319d365-4deb-4a02-8dc7-76ed3dfc432f None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:33.053696 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319d365-4deb-4a02-8dc7-76ed3dfc432f None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:33.054001 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319d365-4deb-4a02-8dc7-76ed3dfc432f None admin] PATH_INFO: `/v3/users/4e51845ddf1f4063a58d7fd540f4aacf` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:33.055217 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8319d365-4deb-4a02-8dc7-76ed3dfc432f None admin] RBAC: Authorizing `identity:get_user(user_id=4e51845ddf1f4063a58d7fd540f4aacf)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:33.056660 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8319d365-4deb-4a02-8dc7-76ed3dfc432f None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:33.058575 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 595/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 16:00:32 2022] GET /identity/v3/users/4e51845ddf1f4063a58d7fd540f4aacf => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:33.064361 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e740525-5e82-45f6-bd8d-e35ec6ebafec None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:33.164891 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e740525-5e82-45f6-bd8d-e35ec6ebafec None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:33.197105 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ee540-8e7d-43ad-800e-eacaf6b6ed96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:33.198509 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ee540-8e7d-43ad-800e-eacaf6b6ed96 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:33.198795 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ee540-8e7d-43ad-800e-eacaf6b6ed96 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:33.199067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ee540-8e7d-43ad-800e-eacaf6b6ed96 None admin] PATH_INFO: `/v3/projects/c7d1258df72d466da6fe2f7910f0754b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:33.200259 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ee540-8e7d-43ad-800e-eacaf6b6ed96 None admin] RBAC: Authorizing `identity:get_project(project_id=c7d1258df72d466da6fe2f7910f0754b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:33.201691 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ee540-8e7d-43ad-800e-eacaf6b6ed96 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:33.203421 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 596/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 16:00:33 2022] GET /identity/v3/projects/c7d1258df72d466da6fe2f7910f0754b => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:33.209399 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319d365-4deb-4a02-8dc7-76ed3dfc432f None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:33.308610 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319d365-4deb-4a02-8dc7-76ed3dfc432f None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:33.330420 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:33.342737 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e257156-11f5-4789-8654-2a0bd44198a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:33.343622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e257156-11f5-4789-8654-2a0bd44198a5 None admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:33.343901 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e257156-11f5-4789-8654-2a0bd44198a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:33.344173 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e257156-11f5-4789-8654-2a0bd44198a5 None admin] PATH_INFO: `/v3/projects/c7d1258df72d466da6fe2f7910f0754b/users/4e51845ddf1f4063a58d7fd540f4aacf/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:33.346163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e257156-11f5-4789-8654-2a0bd44198a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7d1258df72d466da6fe2f7910f0754b, user_id=4e51845ddf1f4063a58d7fd540f4aacf, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:33.347618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e257156-11f5-4789-8654-2a0bd44198a5 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:33.360163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 596/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 16:00:33 2022] PUT /identity/v3/projects/c7d1258df72d466da6fe2f7910f0754b/users/4e51845ddf1f4063a58d7fd540f4aacf/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:00:35.478981 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974dbdf-0a4c-491d-952a-fe1dd76c0d07 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:35.479689 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974dbdf-0a4c-491d-952a-fe1dd76c0d07 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:35.479689 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974dbdf-0a4c-491d-952a-fe1dd76c0d07 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:35.480906 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 597/1193] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:35.486126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb191d56-5d4d-484b-a8e5-5b35868e19b3 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:35.486535 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb191d56-5d4d-484b-a8e5-5b35868e19b3 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:35.486762 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb191d56-5d4d-484b-a8e5-5b35868e19b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:35.503732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-fb191d56-5d4d-484b-a8e5-5b35868e19b3 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:35.504922 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-fb191d56-5d4d-484b-a8e5-5b35868e19b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:35.571211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 597/1194] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:35.596212 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974dbdf-0a4c-491d-952a-fe1dd76c0d07 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:35.640118 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:35.701566 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974dbdf-0a4c-491d-952a-fe1dd76c0d07 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:35.736199 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b9e7f-41ca-45fa-8190-a15cf03d6607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:35.736914 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b9e7f-41ca-45fa-8190-a15cf03d6607 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:35.737193 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b9e7f-41ca-45fa-8190-a15cf03d6607 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:35.737463 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b9e7f-41ca-45fa-8190-a15cf03d6607 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:35.747831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b9e7f-41ca-45fa-8190-a15cf03d6607 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:00:35.755314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-566b9e7f-41ca-45fa-8190-a15cf03d6607 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:00:35.755314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:35.755314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:35.755314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:35.755314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:35.755314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:35.755314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:35.755314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:35.755314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:35.755314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:35.755314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:35.755314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:35.755314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:00:35.755314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:00:35.755314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:00:35.755314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:00:35.756410 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:00:35.756410 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:00:35.756410 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:00:35.756410 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:00:35.756410 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:35.756410 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:35.756410 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:35.756410 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:35.756410 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:35.756410 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:35.756410 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:35.756410 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:35.756410 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:35.756410 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:35.756410 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:35.756410 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:35.757621 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:35.757621 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:35.757621 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:35.757621 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:35.757621 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:00:35.757621 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:00:35.757621 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:00:35.757621 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:00:35.757621 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:00:35.757621 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:00:35.757621 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:00:35.757621 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 16:00:35.757621 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 598/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 16:00:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:35.763004 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb191d56-5d4d-484b-a8e5-5b35868e19b3 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:35.865004 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb191d56-5d4d-484b-a8e5-5b35868e19b3 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:35.878249 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:35.901470 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82d961-0176-4826-8373-4f7c88779c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:35.902177 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82d961-0176-4826-8373-4f7c88779c68 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:35.902464 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82d961-0176-4826-8373-4f7c88779c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:35.902743 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82d961-0176-4826-8373-4f7c88779c68 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:35.903494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da82d961-0176-4826-8373-4f7c88779c68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:35.903797 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da82d961-0176-4826-8373-4f7c88779c68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:35.905186 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da82d961-0176-4826-8373-4f7c88779c68 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:35.913328 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 598/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 16:00:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:35.919462 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b9e7f-41ca-45fa-8190-a15cf03d6607 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:36.016540 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:36.020910 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b9e7f-41ca-45fa-8190-a15cf03d6607 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:36.054780 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd01d8c-1585-40de-b0b3-8cc16bf3a9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:36.055647 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd01d8c-1585-40de-b0b3-8cc16bf3a9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:36.055927 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd01d8c-1585-40de-b0b3-8cc16bf3a9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:36.056197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd01d8c-1585-40de-b0b3-8cc16bf3a9a3 None admin] PATH_INFO: `/v3/users/4e51845ddf1f4063a58d7fd540f4aacf` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:36.057419 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd01d8c-1585-40de-b0b3-8cc16bf3a9a3 None admin] RBAC: Authorizing `identity:get_user(user_id=4e51845ddf1f4063a58d7fd540f4aacf)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:36.059014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd01d8c-1585-40de-b0b3-8cc16bf3a9a3 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:36.060654 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 599/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 16:00:35 2022] GET /identity/v3/users/4e51845ddf1f4063a58d7fd540f4aacf => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:36.066463 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82d961-0176-4826-8373-4f7c88779c68 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:36.166754 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82d961-0176-4826-8373-4f7c88779c68 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:36.200621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d7407-116a-4d58-8936-bcb134c3d5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:36.201270 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d7407-116a-4d58-8936-bcb134c3d5d9 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:36.201558 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d7407-116a-4d58-8936-bcb134c3d5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:36.201836 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d7407-116a-4d58-8936-bcb134c3d5d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:36.242996 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:36.269730 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d7407-116a-4d58-8936-bcb134c3d5d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:36.271163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d7407-116a-4d58-8936-bcb134c3d5d9 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:36.367061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 599/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 16:00:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:00:36.373498 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd01d8c-1585-40de-b0b3-8cc16bf3a9a3 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:36.474163 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd01d8c-1585-40de-b0b3-8cc16bf3a9a3 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:36.508082 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ff0e4f-4272-43b4-a6fb-ff9c1679610f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:36.509195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ff0e4f-4272-43b4-a6fb-ff9c1679610f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:36.509480 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ff0e4f-4272-43b4-a6fb-ff9c1679610f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:36.509757 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ff0e4f-4272-43b4-a6fb-ff9c1679610f None admin] PATH_INFO: `/v3/projects/c7d1258df72d466da6fe2f7910f0754b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:36.511797 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ff0e4f-4272-43b4-a6fb-ff9c1679610f None admin] RBAC: Authorizing `identity:get_project(project_id=c7d1258df72d466da6fe2f7910f0754b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:36.513225 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ff0e4f-4272-43b4-a6fb-ff9c1679610f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:36.514960 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 600/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 16:00:36 2022] GET /identity/v3/projects/c7d1258df72d466da6fe2f7910f0754b => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:36.521130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d7407-116a-4d58-8936-bcb134c3d5d9 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:36.595640 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:36.625244 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d7407-116a-4d58-8936-bcb134c3d5d9 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:36.660658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f975fef3-e1fe-4e6d-a077-18cf32d1fa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:36.661307 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975fef3-e1fe-4e6d-a077-18cf32d1fa4d None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:36.661594 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975fef3-e1fe-4e6d-a077-18cf32d1fa4d None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:36.661872 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975fef3-e1fe-4e6d-a077-18cf32d1fa4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:36.662732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f975fef3-e1fe-4e6d-a077-18cf32d1fa4d None admin] RBAC: Adding query filter params (role.id=bbbc77ec645d4ff5a8bf29ce0377f578, scope.project.id=c7d1258df72d466da6fe2f7910f0754b, user.id=4e51845ddf1f4063a58d7fd540f4aacf) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:36.663047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f975fef3-e1fe-4e6d-a077-18cf32d1fa4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:36.664485 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f975fef3-e1fe-4e6d-a077-18cf32d1fa4d None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:36.670507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 600/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 16:00:36 2022] GET /identity/v3/role_assignments?user.id=4e51845ddf1f4063a58d7fd540f4aacf&scope.project.id=c7d1258df72d466da6fe2f7910f0754b&role.id=bbbc77ec645d4ff5a8bf29ce0377f578 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:46.503837 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=83 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:46.667763 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=31 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:35.320451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123560cb-6817-4310-9f6b-725792e2b26f None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:35.321635 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123560cb-6817-4310-9f6b-725792e2b26f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:35.321977 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123560cb-6817-4310-9f6b-725792e2b26f None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:35.323484 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 601/1201] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:01:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:01:35.329363 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d338c4-38ea-4370-8782-646e0c21743a None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:35.329913 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d338c4-38ea-4370-8782-646e0c21743a None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:35.329913 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d338c4-38ea-4370-8782-646e0c21743a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:35.349803 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-f8d338c4-38ea-4370-8782-646e0c21743a None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:01:35.351510 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-f8d338c4-38ea-4370-8782-646e0c21743a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:01:35.450908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 601/1202] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:01:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:01:35.478206 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123560cb-6817-4310-9f6b-725792e2b26f None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:35.630504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123560cb-6817-4310-9f6b-725792e2b26f None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:35.681639 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665913a-644b-44f3-8d39-fb0110c10616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:35.682486 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665913a-644b-44f3-8d39-fb0110c10616 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:35.682764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665913a-644b-44f3-8d39-fb0110c10616 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:35.683038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665913a-644b-44f3-8d39-fb0110c10616 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:35.697053 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0665913a-644b-44f3-8d39-fb0110c10616 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:35.699121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0665913a-644b-44f3-8d39-fb0110c10616 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:35.724776 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-0665913a-644b-44f3-8d39-fb0110c10616 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:01:35.724776 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:35.724776 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:35.724776 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:35.724776 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:35.724776 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:35.724776 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:35.724776 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:35.724776 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:35.724776 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:35.724776 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:35.724776 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:35.724776 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:01:35.724776 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:01:35.724776 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:01:35.726812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:01:35.726812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:35.726812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:35.726812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:35.726812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:35.726812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:35.726812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:35.726812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:35.726812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:35.726812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:35.726812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:35.726812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:35.726812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:35.726812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:35.726812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:35.726812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:35.726812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:35.727879 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:01:35.727879 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:01:35.727879 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:01:35.727879 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 16:01:35.727879 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 602/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 16:01:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:01:35.732973 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d338c4-38ea-4370-8782-646e0c21743a None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:35.872486 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d338c4-38ea-4370-8782-646e0c21743a None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:35.912882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:35.923163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb48f3e-79e3-4a6b-9762-16394aaf25af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:35.923936 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb48f3e-79e3-4a6b-9762-16394aaf25af None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:35.924338 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb48f3e-79e3-4a6b-9762-16394aaf25af None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:35.924621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb48f3e-79e3-4a6b-9762-16394aaf25af None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:35.925355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb48f3e-79e3-4a6b-9762-16394aaf25af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:35.925665 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb48f3e-79e3-4a6b-9762-16394aaf25af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:35.927346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb48f3e-79e3-4a6b-9762-16394aaf25af None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:35.943817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 602/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 16:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:35.949332 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665913a-644b-44f3-8d39-fb0110c10616 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:36.027304 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:36.094617 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665913a-644b-44f3-8d39-fb0110c10616 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:36.135900 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377eaf83-f8bb-49b2-a52a-60d20f797d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:36.136627 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377eaf83-f8bb-49b2-a52a-60d20f797d21 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:36.136919 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377eaf83-f8bb-49b2-a52a-60d20f797d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:36.137190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377eaf83-f8bb-49b2-a52a-60d20f797d21 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:36.138039 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377eaf83-f8bb-49b2-a52a-60d20f797d21 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:36.139403 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377eaf83-f8bb-49b2-a52a-60d20f797d21 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:36.162516 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 603/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 16:01:35 2022] POST /identity/v3/users => generated 251 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:01:38.549220 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c935fe1-6f1f-4ba3-be2f-3179428e554c None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:38.550046 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c935fe1-6f1f-4ba3-be2f-3179428e554c None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:38.550046 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c935fe1-6f1f-4ba3-be2f-3179428e554c None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:38.551247 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 603/1206] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:01:38.557429 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2928095-8dd2-4442-a756-0df19520d3e4 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:38.557992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2928095-8dd2-4442-a756-0df19520d3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:38.558097 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2928095-8dd2-4442-a756-0df19520d3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:38.580351 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e2928095-8dd2-4442-a756-0df19520d3e4 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:01:38.581563 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e2928095-8dd2-4442-a756-0df19520d3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:01:38.666693 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 604/1207] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:01:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:01:38.691046 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c935fe1-6f1f-4ba3-be2f-3179428e554c None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:38.817516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c935fe1-6f1f-4ba3-be2f-3179428e554c None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:38.860949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac732591-2dd7-4cdd-850a-0acae9eaf993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:38.860949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac732591-2dd7-4cdd-850a-0acae9eaf993 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:38.860949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac732591-2dd7-4cdd-850a-0acae9eaf993 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:38.860949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac732591-2dd7-4cdd-850a-0acae9eaf993 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:38.875509 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac732591-2dd7-4cdd-850a-0acae9eaf993 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:01:38.884592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-ac732591-2dd7-4cdd-850a-0acae9eaf993 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:01:38.884592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:38.884592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:38.884592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:38.884592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:38.884592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:38.884592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:38.884592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:38.884592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:38.884592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:38.884592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:38.884592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:38.884592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:01:38.884592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:01:38.884592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:01:38.884592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:01:38.885611 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:01:38.885611 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:01:38.885611 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:01:38.885611 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:01:38.885611 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:38.885611 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:38.885611 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:38.885611 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:38.885611 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:38.885611 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:38.885611 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:38.885611 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:38.885611 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:38.885611 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:38.885611 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:38.885611 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:38.886604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:38.886604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:38.886604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:38.886604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:38.886604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:01:38.886604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:01:38.886604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:01:38.886604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:01:38.886604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:01:38.886604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:01:38.886604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:01:38.886604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 16:01:38.886604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 604/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 16:01:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:01:38.892337 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2928095-8dd2-4442-a756-0df19520d3e4 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:38.938572 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:39.031170 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2928095-8dd2-4442-a756-0df19520d3e4 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:39.071464 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b41332-2f0b-4169-bcdd-d50385d9f182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:39.071464 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b41332-2f0b-4169-bcdd-d50385d9f182 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:39.071464 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b41332-2f0b-4169-bcdd-d50385d9f182 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:39.071464 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b41332-2f0b-4169-bcdd-d50385d9f182 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:39.071879 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b41332-2f0b-4169-bcdd-d50385d9f182 None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:39.071879 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b41332-2f0b-4169-bcdd-d50385d9f182 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:39.072627 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b41332-2f0b-4169-bcdd-d50385d9f182 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:39.081494 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 605/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 16:01:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:39.089003 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac732591-2dd7-4cdd-850a-0acae9eaf993 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:39.150832 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:39.209668 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac732591-2dd7-4cdd-850a-0acae9eaf993 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:39.251530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60eb04e-225f-48b1-a53c-239c23496d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:39.252677 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60eb04e-225f-48b1-a53c-239c23496d8a None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:39.253158 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60eb04e-225f-48b1-a53c-239c23496d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:39.253622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60eb04e-225f-48b1-a53c-239c23496d8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:39.265572 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60eb04e-225f-48b1-a53c-239c23496d8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:39.267089 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60eb04e-225f-48b1-a53c-239c23496d8a None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:39.286712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-c60eb04e-225f-48b1-a53c-239c23496d8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:01:39.286712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:39.286712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:39.286712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:39.286712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:39.286712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:39.286712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:39.286712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:39.286712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:39.286712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:39.286712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:39.286712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:39.286712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:01:39.286712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:01:39.286712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:01:39.287718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:01:39.287718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:39.287718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:39.287718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:39.287718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:39.287718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:39.287718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:39.287718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:39.287718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:39.287718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:39.287718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:39.287718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:39.287718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:39.287718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:39.287718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:39.287718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:39.287718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:39.288616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:01:39.288616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:01:39.288616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:01:39.288616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 16:01:39.288616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 605/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 16:01:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:01:39.294718 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b41332-2f0b-4169-bcdd-d50385d9f182 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:39.414113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:39.417321 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b41332-2f0b-4169-bcdd-d50385d9f182 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:39.460716 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b42ac6-283a-4ab9-9121-77a9d43c86be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:39.462959 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b42ac6-283a-4ab9-9121-77a9d43c86be None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:39.462959 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b42ac6-283a-4ab9-9121-77a9d43c86be None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:39.462959 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b42ac6-283a-4ab9-9121-77a9d43c86be None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:39.463466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b42ac6-283a-4ab9-9121-77a9d43c86be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:39.463772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b42ac6-283a-4ab9-9121-77a9d43c86be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:39.465152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b42ac6-283a-4ab9-9121-77a9d43c86be None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:39.482637 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 606/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 16:01:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:39.488305 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60eb04e-225f-48b1-a53c-239c23496d8a None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:39.610712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60eb04e-225f-48b1-a53c-239c23496d8a None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:39.612587 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:39.658131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737832ee-dafe-4bbe-bf15-5e9785d8d9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:39.658988 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737832ee-dafe-4bbe-bf15-5e9785d8d9c2 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:39.659292 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737832ee-dafe-4bbe-bf15-5e9785d8d9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:39.659525 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737832ee-dafe-4bbe-bf15-5e9785d8d9c2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:39.669975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737832ee-dafe-4bbe-bf15-5e9785d8d9c2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:39.671605 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737832ee-dafe-4bbe-bf15-5e9785d8d9c2 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:39.688457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-737832ee-dafe-4bbe-bf15-5e9785d8d9c2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:01:39.688457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:39.688457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:39.688457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:39.688457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:39.688457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:39.688457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:39.688457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:39.688457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:39.688457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:39.688457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:39.688457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:39.688457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:01:39.688457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:01:39.688457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:01:39.688457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:01:39.689622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:39.689622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:39.689622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:01:39.689622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:01:39.689622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:01:39.689622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:01:39.689622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:39.689622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:39.689622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:39.689622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:39.689622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:39.689622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:39.689622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:39.689622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:39.689622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:39.689622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:39.689622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:39.689622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:39.690923 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:39.690923 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:39.690923 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:01:39.690923 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:01:39.690923 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:01:39.690923 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:01:39.690923 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:01:39.690923 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:01:39.690923 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:01:39.690923 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 16:01:39.694098 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 606/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 16:01:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:01:39.696550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b42ac6-283a-4ab9-9121-77a9d43c86be None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:39.828521 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b42ac6-283a-4ab9-9121-77a9d43c86be None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:39.885882 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7132c4-3ee3-4327-9b31-3769d934f4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:39.886655 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7132c4-3ee3-4327-9b31-3769d934f4c3 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:39.886955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7132c4-3ee3-4327-9b31-3769d934f4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:39.887228 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7132c4-3ee3-4327-9b31-3769d934f4c3 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:39.888081 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7132c4-3ee3-4327-9b31-3769d934f4c3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:39.888394 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7132c4-3ee3-4327-9b31-3769d934f4c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:39.889857 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7132c4-3ee3-4327-9b31-3769d934f4c3 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:39.915957 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 607/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jun 16 16:01:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:39.922211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737832ee-dafe-4bbe-bf15-5e9785d8d9c2 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:40.050231 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737832ee-dafe-4bbe-bf15-5e9785d8d9c2 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:40.091874 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579be69a-56b0-4380-aac8-5cbc00a47338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:40.092538 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579be69a-56b0-4380-aac8-5cbc00a47338 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:40.093221 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579be69a-56b0-4380-aac8-5cbc00a47338 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:40.093221 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579be69a-56b0-4380-aac8-5cbc00a47338 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:40.131074 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:40.177329 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579be69a-56b0-4380-aac8-5cbc00a47338 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:40.178879 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579be69a-56b0-4380-aac8-5cbc00a47338 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:40.288589 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 607/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 16:01:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:01:40.295434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7132c4-3ee3-4327-9b31-3769d934f4c3 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:40.318724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:40.417452 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7132c4-3ee3-4327-9b31-3769d934f4c3 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:40.455631 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda1536-4de5-47e7-8165-56e68b519a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:40.456575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda1536-4de5-47e7-8165-56e68b519a8f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:40.457007 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda1536-4de5-47e7-8165-56e68b519a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:40.457417 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda1536-4de5-47e7-8165-56e68b519a8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:40.471016 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbda1536-4de5-47e7-8165-56e68b519a8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:40.472692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbda1536-4de5-47e7-8165-56e68b519a8f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:40.493430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-bbda1536-4de5-47e7-8165-56e68b519a8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:01:40.493430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:40.493430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:40.493430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:40.493430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:40.493430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:40.493430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:40.493430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:40.493430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:40.493430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:40.493430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:40.493430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:40.493430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:01:40.493430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:01:40.493430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:01:40.494699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:01:40.494699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:40.494699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:40.494699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:40.494699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:40.494699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:40.494699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:40.494699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:40.494699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:40.494699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:40.494699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:40.494699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:40.494699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:40.494699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:40.494699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:40.494699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:40.494699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:40.495777 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:01:40.495777 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:01:40.495777 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:01:40.495777 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 16:01:40.495777 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 608/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 16:01:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:01:40.500930 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579be69a-56b0-4380-aac8-5cbc00a47338 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:40.573762 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:40.622173 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579be69a-56b0-4380-aac8-5cbc00a47338 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:40.660297 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81567cf9-b771-439d-95ba-70acec6851d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:40.661119 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81567cf9-b771-439d-95ba-70acec6851d5 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:40.661527 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81567cf9-b771-439d-95ba-70acec6851d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:40.661904 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81567cf9-b771-439d-95ba-70acec6851d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:40.662749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81567cf9-b771-439d-95ba-70acec6851d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:40.663159 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81567cf9-b771-439d-95ba-70acec6851d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:40.665019 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81567cf9-b771-439d-95ba-70acec6851d5 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:40.683448 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 608/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 16:01:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:40.685054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda1536-4de5-47e7-8165-56e68b519a8f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:40.772194 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:40.819745 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda1536-4de5-47e7-8165-56e68b519a8f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:40.869771 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b5b48-49e8-4db2-9556-851494134325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:40.870630 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b5b48-49e8-4db2-9556-851494134325 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:40.870918 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b5b48-49e8-4db2-9556-851494134325 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:40.871196 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b5b48-49e8-4db2-9556-851494134325 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:40.878962 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88b5b48-49e8-4db2-9556-851494134325 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:40.880531 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88b5b48-49e8-4db2-9556-851494134325 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:40.896466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-c88b5b48-49e8-4db2-9556-851494134325 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:01:40.896466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:40.896466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:40.896466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:40.896466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:40.896466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:40.896466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:40.896466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:40.896466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:40.896466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:40.896466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:40.896466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:40.896466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:01:40.896466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:01:40.896466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:01:40.897692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:01:40.897692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:40.897692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:40.897692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:40.897692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:40.897692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:40.897692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:40.897692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:40.897692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:40.897692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:40.897692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:40.897692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:40.897692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:40.897692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:40.897692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:40.897692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:40.897692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:40.899175 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:01:40.899175 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:01:40.899175 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:01:40.899175 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:01:40.899175 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:01:40.899175 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:01:40.899175 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:01:40.899175 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 16:01:40.899175 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 609/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 16:01:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:01:40.904751 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81567cf9-b771-439d-95ba-70acec6851d5 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:41.027855 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81567cf9-b771-439d-95ba-70acec6851d5 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:41.065193 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:41.074498 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8735e5-6c0a-43d9-95d3-bc0e46b3af94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:41.075553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8735e5-6c0a-43d9-95d3-bc0e46b3af94 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:41.075919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8735e5-6c0a-43d9-95d3-bc0e46b3af94 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:41.076296 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8735e5-6c0a-43d9-95d3-bc0e46b3af94 None admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:41.077252 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8735e5-6c0a-43d9-95d3-bc0e46b3af94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:41.077824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8735e5-6c0a-43d9-95d3-bc0e46b3af94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:41.079873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8735e5-6c0a-43d9-95d3-bc0e46b3af94 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:41.094959 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 609/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 16:01:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:41.101336 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b5b48-49e8-4db2-9556-851494134325 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:41.234353 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b5b48-49e8-4db2-9556-851494134325 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:41.274645 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:41.284438 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c8f72-1b67-463c-a999-14941fdafb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:41.285407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c8f72-1b67-463c-a999-14941fdafb62 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:41.285881 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c8f72-1b67-463c-a999-14941fdafb62 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:41.286393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c8f72-1b67-463c-a999-14941fdafb62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:41.287897 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c8f72-1b67-463c-a999-14941fdafb62 None admin] RBAC: Adding query filter params (role.id=cfc67444e9944ffc87173be8c9245108, scope.project.id=b1177a85a0de4276898daf4a24ff069f, user.id=36b63614297346e29ff276eab444fe5b) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:41.288361 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c8f72-1b67-463c-a999-14941fdafb62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:41.290859 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c8f72-1b67-463c-a999-14941fdafb62 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:41.300918 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 610/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 16:01:41 2022] GET /identity/v3/role_assignments?user.id=36b63614297346e29ff276eab444fe5b&scope.project.id=b1177a85a0de4276898daf4a24ff069f&role.id=cfc67444e9944ffc87173be8c9245108 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:43.553529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec016e9-5925-4708-8ead-85a9bbcf3602 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:43.554264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec016e9-5925-4708-8ead-85a9bbcf3602 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:43.554264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec016e9-5925-4708-8ead-85a9bbcf3602 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:43.556118 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 610/1220] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:01:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:01:43.562151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441bc35-bb3c-4be5-9743-fc1a062f7867 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:43.562696 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441bc35-bb3c-4be5-9743-fc1a062f7867 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:43.562696 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441bc35-bb3c-4be5-9743-fc1a062f7867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:43.581400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-a441bc35-bb3c-4be5-9743-fc1a062f7867 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:01:43.582643 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a441bc35-bb3c-4be5-9743-fc1a062f7867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:01:43.656579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 611/1221] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:01:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:01:43.681527 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec016e9-5925-4708-8ead-85a9bbcf3602 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:43.807211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec016e9-5925-4708-8ead-85a9bbcf3602 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:43.852435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c23ed-58c1-4d43-baa6-4491de52bc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:43.853237 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c23ed-58c1-4d43-baa6-4491de52bc2b None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:43.853522 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c23ed-58c1-4d43-baa6-4491de52bc2b None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:43.853808 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c23ed-58c1-4d43-baa6-4491de52bc2b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:43.864333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279c23ed-58c1-4d43-baa6-4491de52bc2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:01:43.872538 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-279c23ed-58c1-4d43-baa6-4491de52bc2b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:01:43.872538 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:43.872538 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:43.872538 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:43.872538 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:43.872538 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:43.872538 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:43.872538 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:43.872538 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:43.872538 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:43.872538 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:43.872538 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:43.872538 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:01:43.872538 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:01:43.872538 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:01:43.872538 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:01:43.873817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:01:43.873817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:01:43.873817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:01:43.873817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:01:43.873817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:43.873817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:43.873817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:43.873817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:43.873817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:43.873817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:43.873817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:43.873817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:43.873817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:43.873817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:43.873817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:43.873817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:43.875166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:43.875166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:43.875166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:43.875166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:43.875166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:01:43.875166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:01:43.875166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:01:43.875166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:01:43.875166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:01:43.875166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:01:43.875166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:01:43.875166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 16:01:43.878099 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 611/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 16:01:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:01:43.879901 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441bc35-bb3c-4be5-9743-fc1a062f7867 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:43.997865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441bc35-bb3c-4be5-9743-fc1a062f7867 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:44.023803 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:44.036639 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18841842-586e-4a3b-b6d4-189421ae9809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:44.037325 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18841842-586e-4a3b-b6d4-189421ae9809 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:44.037613 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18841842-586e-4a3b-b6d4-189421ae9809 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:44.037894 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18841842-586e-4a3b-b6d4-189421ae9809 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:44.038682 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18841842-586e-4a3b-b6d4-189421ae9809 None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:44.039023 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18841842-586e-4a3b-b6d4-189421ae9809 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:44.040927 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18841842-586e-4a3b-b6d4-189421ae9809 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:44.049418 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 612/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 16:01:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:44.056500 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c23ed-58c1-4d43-baa6-4491de52bc2b None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:44.104176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:44.169278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c23ed-58c1-4d43-baa6-4491de52bc2b None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:44.212256 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56505d8e-10c7-4307-be87-152b09f3fb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:44.213012 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56505d8e-10c7-4307-be87-152b09f3fb29 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:44.213302 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56505d8e-10c7-4307-be87-152b09f3fb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:44.213582 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56505d8e-10c7-4307-be87-152b09f3fb29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:44.224832 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56505d8e-10c7-4307-be87-152b09f3fb29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:44.226327 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56505d8e-10c7-4307-be87-152b09f3fb29 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:44.249656 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-56505d8e-10c7-4307-be87-152b09f3fb29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:01:44.249656 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:44.249656 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:44.249656 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:44.249656 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:44.249656 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:44.249656 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:44.249656 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:44.249656 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:44.249656 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:44.249656 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:44.249656 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:44.249656 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:01:44.249656 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:01:44.249656 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:01:44.251077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:01:44.251077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:44.251077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:44.251077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:44.251077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:44.251077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:44.251077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:44.251077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:44.251077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:44.251077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:44.251077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:44.251077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:44.251077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:44.251077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:44.251077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:44.251077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:44.251077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:44.252053 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:01:44.252053 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:01:44.252053 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:01:44.252053 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 16:01:44.252053 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 612/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 16:01:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:01:44.257234 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18841842-586e-4a3b-b6d4-189421ae9809 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:44.373832 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18841842-586e-4a3b-b6d4-189421ae9809 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:44.413339 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d516ba4-4e48-40fd-b6da-a08ae315cf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:44.414229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d516ba4-4e48-40fd-b6da-a08ae315cf12 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:44.415626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d516ba4-4e48-40fd-b6da-a08ae315cf12 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:44.415626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d516ba4-4e48-40fd-b6da-a08ae315cf12 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:44.416354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d516ba4-4e48-40fd-b6da-a08ae315cf12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:44.416719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d516ba4-4e48-40fd-b6da-a08ae315cf12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:44.418214 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d516ba4-4e48-40fd-b6da-a08ae315cf12 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:44.432910 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 613/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 16:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:44.439087 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56505d8e-10c7-4307-be87-152b09f3fb29 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:44.557109 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:44.564672 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56505d8e-10c7-4307-be87-152b09f3fb29 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:44.600899 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dded2-c661-498a-ab22-469c577e011d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:44.602678 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dded2-c661-498a-ab22-469c577e011d None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:44.603071 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dded2-c661-498a-ab22-469c577e011d None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:44.603396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dded2-c661-498a-ab22-469c577e011d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:44.612083 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000dded2-c661-498a-ab22-469c577e011d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:44.613752 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000dded2-c661-498a-ab22-469c577e011d None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:44.630070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-000dded2-c661-498a-ab22-469c577e011d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:01:44.630070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:44.630070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:44.630070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:44.630070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:44.630070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:44.630070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:44.630070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:44.630070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:44.630070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:44.630070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:44.630070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:44.630070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:01:44.630070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:01:44.630070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:01:44.630070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:01:44.631085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:44.631085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:44.631085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:01:44.631085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:01:44.631085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:01:44.631085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:01:44.631085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:44.631085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:44.631085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:44.631085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:44.631085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:44.631085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:44.631085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:44.631085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:44.631085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:44.631085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:44.631085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:44.631085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:44.632038 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:44.632038 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:44.632038 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:01:44.632038 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:01:44.632038 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:01:44.632038 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:01:44.632038 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:01:44.632038 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:01:44.632038 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:01:44.632038 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 16:01:44.632038 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 613/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 16:01:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:01:44.638194 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d516ba4-4e48-40fd-b6da-a08ae315cf12 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:44.699278 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:44.763154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d516ba4-4e48-40fd-b6da-a08ae315cf12 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:44.807362 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdc401-41a6-4756-96a6-10e1413b3610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:44.808147 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdc401-41a6-4756-96a6-10e1413b3610 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:44.808434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdc401-41a6-4756-96a6-10e1413b3610 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:44.808710 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdc401-41a6-4756-96a6-10e1413b3610 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:44.809590 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcdc401-41a6-4756-96a6-10e1413b3610 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:44.809902 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcdc401-41a6-4756-96a6-10e1413b3610 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:44.811507 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcdc401-41a6-4756-96a6-10e1413b3610 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:44.836150 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 614/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jun 16 16:01:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:44.837948 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dded2-c661-498a-ab22-469c577e011d None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:44.963452 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dded2-c661-498a-ab22-469c577e011d None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:45.005981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caee99b-e731-4e42-bae1-9b5728f93610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:45.006811 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caee99b-e731-4e42-bae1-9b5728f93610 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:45.007166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caee99b-e731-4e42-bae1-9b5728f93610 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:45.007585 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caee99b-e731-4e42-bae1-9b5728f93610 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:45.021824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caee99b-e731-4e42-bae1-9b5728f93610 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:45.024170 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caee99b-e731-4e42-bae1-9b5728f93610 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:45.052644 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-9caee99b-e731-4e42-bae1-9b5728f93610 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:01:45.052644 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:45.052644 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:45.052644 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:45.052644 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:45.052644 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:45.052644 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:45.052644 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:45.052644 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:45.052644 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:45.052644 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:45.052644 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:45.052644 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:01:45.052644 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:01:45.052644 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:01:45.053833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:01:45.053833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:45.053833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:45.053833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:45.053833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:45.053833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:45.053833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:45.053833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:45.053833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:45.053833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:45.053833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:45.053833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:45.053833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:45.053833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:45.053833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:45.053833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:45.053833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:45.056101 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:01:45.056101 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:01:45.056101 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:01:45.056101 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 16:01:45.056101 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 614/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 16:01:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:01:45.062698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdc401-41a6-4756-96a6-10e1413b3610 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:45.190987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdc401-41a6-4756-96a6-10e1413b3610 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:45.194016 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:45.236329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d202c8-e73a-4e93-a4a6-cc37b6bf787f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:45.237048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d202c8-e73a-4e93-a4a6-cc37b6bf787f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:45.237390 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d202c8-e73a-4e93-a4a6-cc37b6bf787f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:45.237703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d202c8-e73a-4e93-a4a6-cc37b6bf787f None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:45.238790 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d202c8-e73a-4e93-a4a6-cc37b6bf787f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:45.239193 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d202c8-e73a-4e93-a4a6-cc37b6bf787f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:45.240899 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d202c8-e73a-4e93-a4a6-cc37b6bf787f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:45.255803 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 615/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 16:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:45.263404 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caee99b-e731-4e42-bae1-9b5728f93610 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:45.273101 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:45.380234 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caee99b-e731-4e42-bae1-9b5728f93610 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:45.418977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523beeb-dbaa-4258-9977-ed27c044e2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:45.419846 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523beeb-dbaa-4258-9977-ed27c044e2d2 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:45.420129 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523beeb-dbaa-4258-9977-ed27c044e2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:45.420407 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523beeb-dbaa-4258-9977-ed27c044e2d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:45.427807 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523beeb-dbaa-4258-9977-ed27c044e2d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:45.429086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523beeb-dbaa-4258-9977-ed27c044e2d2 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:45.443777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-b523beeb-dbaa-4258-9977-ed27c044e2d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:01:45.443777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:45.443777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:45.443777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:45.443777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:45.443777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:45.443777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:45.443777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:45.443777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:45.443777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:45.443777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:45.443777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:45.443777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:01:45.443777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:01:45.443777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:01:45.445359 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:01:45.445359 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:45.445359 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:45.445359 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:45.445359 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:45.445359 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:45.445359 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:45.445359 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:45.445359 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:45.445359 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:45.445359 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:45.445359 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:45.445359 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:45.445359 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:45.445359 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:45.445359 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:45.445359 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:45.446441 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:01:45.446441 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:01:45.446441 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:01:45.446441 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:01:45.446441 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:01:45.446441 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:01:45.446441 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:01:45.446441 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 16:01:45.446441 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 615/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 16:01:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:01:45.451182 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d202c8-e73a-4e93-a4a6-cc37b6bf787f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:45.566868 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d202c8-e73a-4e93-a4a6-cc37b6bf787f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:45.608193 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32bf60-1d53-4b4e-9483-095e29ab2605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:45.609173 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32bf60-1d53-4b4e-9483-095e29ab2605 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:45.609594 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32bf60-1d53-4b4e-9483-095e29ab2605 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:45.610034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32bf60-1d53-4b4e-9483-095e29ab2605 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:45.611047 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32bf60-1d53-4b4e-9483-095e29ab2605 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:45.611396 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32bf60-1d53-4b4e-9483-095e29ab2605 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:45.612866 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32bf60-1d53-4b4e-9483-095e29ab2605 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:45.628573 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 616/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 16:01:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:45.635233 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523beeb-dbaa-4258-9977-ed27c044e2d2 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:45.716399 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:45.760251 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523beeb-dbaa-4258-9977-ed27c044e2d2 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:45.803580 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34609432-4053-4822-8b4c-a9a21383ede2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:45.804474 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34609432-4053-4822-8b4c-a9a21383ede2 None admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:45.804784 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34609432-4053-4822-8b4c-a9a21383ede2 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:45.806229 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34609432-4053-4822-8b4c-a9a21383ede2 None admin] PATH_INFO: `/v3/projects/b1177a85a0de4276898daf4a24ff069f/users/36b63614297346e29ff276eab444fe5b/roles/cfc67444e9944ffc87173be8c9245108` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:45.822465 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34609432-4053-4822-8b4c-a9a21383ede2 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1177a85a0de4276898daf4a24ff069f, user_id=36b63614297346e29ff276eab444fe5b, role_id=cfc67444e9944ffc87173be8c9245108)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:45.823851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34609432-4053-4822-8b4c-a9a21383ede2 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:45.837166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 616/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 16:01:45 2022] PUT /identity/v3/projects/b1177a85a0de4276898daf4a24ff069f/users/36b63614297346e29ff276eab444fe5b/roles/cfc67444e9944ffc87173be8c9245108 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:01:48.086556 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd38367-8c08-464c-a5ab-b8c020b4c1b7 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:48.087234 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd38367-8c08-464c-a5ab-b8c020b4c1b7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:48.087234 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd38367-8c08-464c-a5ab-b8c020b4c1b7 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:48.088580 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 617/1233] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:01:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:01:48.094573 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1737b74-8e69-426f-a7f0-9e8a1d53c21f None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:48.096596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1737b74-8e69-426f-a7f0-9e8a1d53c21f None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:48.096596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1737b74-8e69-426f-a7f0-9e8a1d53c21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:48.115277 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-c1737b74-8e69-426f-a7f0-9e8a1d53c21f None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:01:48.116918 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-c1737b74-8e69-426f-a7f0-9e8a1d53c21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:01:48.205573 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 617/1234] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:01:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:01:48.232032 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd38367-8c08-464c-a5ab-b8c020b4c1b7 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:48.261095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:48.347775 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd38367-8c08-464c-a5ab-b8c020b4c1b7 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:48.387163 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d7d2c-3d5a-49b0-abe1-5916da411ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:48.387999 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d7d2c-3d5a-49b0-abe1-5916da411ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:48.388295 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d7d2c-3d5a-49b0-abe1-5916da411ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:48.388552 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d7d2c-3d5a-49b0-abe1-5916da411ef6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:48.400943 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d7d2c-3d5a-49b0-abe1-5916da411ef6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:01:48.409939 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-f37d7d2c-3d5a-49b0-abe1-5916da411ef6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:01:48.409939 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:48.409939 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:48.409939 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:48.409939 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:48.409939 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:48.409939 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:48.409939 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:48.409939 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:48.409939 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:48.409939 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:48.409939 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:48.409939 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:01:48.409939 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:01:48.409939 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:01:48.409939 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:01:48.411139 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:01:48.411139 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:01:48.411139 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:01:48.411139 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:01:48.411139 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:48.411139 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:48.411139 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:48.411139 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:48.411139 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:48.411139 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:48.411139 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:48.411139 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:48.411139 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:48.411139 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:48.411139 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:48.411139 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:48.418736 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:48.418736 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:48.418736 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:48.418736 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:48.418736 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:01:48.418736 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:01:48.418736 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:01:48.418736 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:01:48.418736 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:01:48.418736 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:01:48.418736 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:01:48.418736 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 16:01:48.418736 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 618/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 16:01:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:01:48.419437 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1737b74-8e69-426f-a7f0-9e8a1d53c21f None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:48.478640 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:48.549825 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1737b74-8e69-426f-a7f0-9e8a1d53c21f None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:48.596853 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb7daea-006f-4d80-ac19-4cee38f88d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:48.598073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb7daea-006f-4d80-ac19-4cee38f88d41 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:48.598498 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb7daea-006f-4d80-ac19-4cee38f88d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:48.598998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb7daea-006f-4d80-ac19-4cee38f88d41 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:48.599977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb7daea-006f-4d80-ac19-4cee38f88d41 None admin] RBAC: Adding query filter params (name=service) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:48.600287 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb7daea-006f-4d80-ac19-4cee38f88d41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:48.601857 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb7daea-006f-4d80-ac19-4cee38f88d41 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:48.611503 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 618/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 16:01:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:48.618326 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d7d2c-3d5a-49b0-abe1-5916da411ef6 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:48.723295 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:48.742983 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d7d2c-3d5a-49b0-abe1-5916da411ef6 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:48.783405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db5baa-2a14-437e-b5ca-ee2f82b8cd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:48.784298 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db5baa-2a14-437e-b5ca-ee2f82b8cd0b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:48.784631 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db5baa-2a14-437e-b5ca-ee2f82b8cd0b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:48.784990 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db5baa-2a14-437e-b5ca-ee2f82b8cd0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:48.797246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db5baa-2a14-437e-b5ca-ee2f82b8cd0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:48.798933 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db5baa-2a14-437e-b5ca-ee2f82b8cd0b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:48.817195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-28db5baa-2a14-437e-b5ca-ee2f82b8cd0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:01:48.817195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:48.817195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:48.817195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:48.817195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:48.817195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:48.817195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:48.817195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:48.817195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:48.817195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:48.817195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:48.817195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:48.817195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:01:48.817195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:01:48.817195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:01:48.818493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:01:48.818493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:48.818493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:48.818493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:48.818493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:48.818493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:48.818493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:48.818493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:48.818493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:48.818493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:48.818493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:48.818493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:48.818493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:48.818493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:48.818493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:48.818493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:48.818493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:48.819452 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:01:48.819452 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:01:48.819452 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:01:48.819452 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 16:01:48.819452 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 619/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 16:01:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:01:48.825003 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb7daea-006f-4d80-ac19-4cee38f88d41 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:48.942943 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb7daea-006f-4d80-ac19-4cee38f88d41 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:48.954610 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:48.992858 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d8758-5fde-45b2-9682-88a2ab2683a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:48.993558 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d8758-5fde-45b2-9682-88a2ab2683a6 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:48.993840 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d8758-5fde-45b2-9682-88a2ab2683a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:48.994141 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d8758-5fde-45b2-9682-88a2ab2683a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:48.994872 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8d8758-5fde-45b2-9682-88a2ab2683a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:48.995182 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8d8758-5fde-45b2-9682-88a2ab2683a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:48.996582 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8d8758-5fde-45b2-9682-88a2ab2683a6 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:49.011036 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 619/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 16:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:49.017862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db5baa-2a14-437e-b5ca-ee2f82b8cd0b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:49.143892 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db5baa-2a14-437e-b5ca-ee2f82b8cd0b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:49.186487 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156deea-b405-45bd-b3f7-d091a918ee0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:49.187484 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156deea-b405-45bd-b3f7-d091a918ee0b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:49.187918 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156deea-b405-45bd-b3f7-d091a918ee0b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:49.188316 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156deea-b405-45bd-b3f7-d091a918ee0b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:49.191501 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:49.198841 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156deea-b405-45bd-b3f7-d091a918ee0b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:49.200370 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156deea-b405-45bd-b3f7-d091a918ee0b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:49.215833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-a156deea-b405-45bd-b3f7-d091a918ee0b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:01:49.215833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:49.215833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:49.215833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:49.215833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:49.215833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:49.215833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:49.215833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:49.215833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:49.215833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:49.215833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:49.215833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:49.215833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:01:49.215833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:01:49.215833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:01:49.215833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:01:49.217239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:49.217239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:49.217239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:01:49.217239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:01:49.217239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:01:49.217239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:01:49.217239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:49.217239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:49.217239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:49.217239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:49.217239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:49.217239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:49.217239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:49.217239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:49.217239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:49.217239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:49.217239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:49.217239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:49.218577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:49.218577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:49.218577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:01:49.218577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:01:49.218577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:01:49.218577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:01:49.218577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:01:49.218577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:01:49.218577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:01:49.218577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 16:01:49.219218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 620/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 16:01:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:01:49.224479 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d8758-5fde-45b2-9682-88a2ab2683a6 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:49.339778 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d8758-5fde-45b2-9682-88a2ab2683a6 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:49.381553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c690d-9e8f-4a41-a7d9-fb417066a6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:49.382303 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c690d-9e8f-4a41-a7d9-fb417066a6ab None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:49.383368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c690d-9e8f-4a41-a7d9-fb417066a6ab None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:49.383945 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c690d-9e8f-4a41-a7d9-fb417066a6ab None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:49.384462 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c690d-9e8f-4a41-a7d9-fb417066a6ab None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:49.385141 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c690d-9e8f-4a41-a7d9-fb417066a6ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:49.386829 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c690d-9e8f-4a41-a7d9-fb417066a6ab None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:49.417426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 620/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jun 16 16:01:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:49.423363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156deea-b405-45bd-b3f7-d091a918ee0b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:49.564122 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156deea-b405-45bd-b3f7-d091a918ee0b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:49.612653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84a904-71e4-4e55-8f9a-05f3bc0d00c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:49.613312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84a904-71e4-4e55-8f9a-05f3bc0d00c2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:49.613618 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84a904-71e4-4e55-8f9a-05f3bc0d00c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:49.613983 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84a904-71e4-4e55-8f9a-05f3bc0d00c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:49.690102 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:49.700193 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84a904-71e4-4e55-8f9a-05f3bc0d00c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:49.702128 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84a904-71e4-4e55-8f9a-05f3bc0d00c2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:49.830243 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 621/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 16:01:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:01:49.836691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c690d-9e8f-4a41-a7d9-fb417066a6ab None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:49.873711 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:49.964489 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c690d-9e8f-4a41-a7d9-fb417066a6ab None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:50.007265 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef45a52-28e7-43ec-bc67-96c4811dbeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:50.008789 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef45a52-28e7-43ec-bc67-96c4811dbeb6 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:50.009124 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef45a52-28e7-43ec-bc67-96c4811dbeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:50.009409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef45a52-28e7-43ec-bc67-96c4811dbeb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:50.026914 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef45a52-28e7-43ec-bc67-96c4811dbeb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:50.028425 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef45a52-28e7-43ec-bc67-96c4811dbeb6 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:50.047482 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-3ef45a52-28e7-43ec-bc67-96c4811dbeb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:01:50.047482 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:50.047482 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:50.047482 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:50.047482 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:50.047482 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:50.047482 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:50.047482 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:50.047482 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:50.047482 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:50.047482 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:50.047482 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:50.047482 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:01:50.047482 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:01:50.047482 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:01:50.049987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:01:50.049987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:50.049987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:50.049987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:50.049987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:50.049987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:50.049987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:50.049987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:50.049987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:50.049987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:50.049987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:50.049987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:50.049987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:50.049987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:50.049987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:50.049987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:50.049987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:50.051121 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:01:50.051121 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:01:50.051121 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:01:50.051121 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 16:01:50.051121 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 621/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 16:01:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:01:50.054622 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84a904-71e4-4e55-8f9a-05f3bc0d00c2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:50.159743 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:50.168025 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84a904-71e4-4e55-8f9a-05f3bc0d00c2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:50.207345 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5183f-c4c2-4d18-b675-935b2d11b8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:50.208078 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5183f-c4c2-4d18-b675-935b2d11b8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:50.208367 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5183f-c4c2-4d18-b675-935b2d11b8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:50.208646 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5183f-c4c2-4d18-b675-935b2d11b8d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:50.209395 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5183f-c4c2-4d18-b675-935b2d11b8d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:50.209748 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5183f-c4c2-4d18-b675-935b2d11b8d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:50.212236 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5183f-c4c2-4d18-b675-935b2d11b8d5 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:50.224604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 622/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 16:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:50.231011 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef45a52-28e7-43ec-bc67-96c4811dbeb6 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:50.342062 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:50.375683 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef45a52-28e7-43ec-bc67-96c4811dbeb6 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:50.424300 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31fdf2-93f8-4ef2-87d4-50c37975de54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:50.425043 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31fdf2-93f8-4ef2-87d4-50c37975de54 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:50.425308 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31fdf2-93f8-4ef2-87d4-50c37975de54 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:50.425631 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31fdf2-93f8-4ef2-87d4-50c37975de54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:50.434488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb31fdf2-93f8-4ef2-87d4-50c37975de54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:50.436252 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb31fdf2-93f8-4ef2-87d4-50c37975de54 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:50.451708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-cb31fdf2-93f8-4ef2-87d4-50c37975de54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:01:50.451708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:50.451708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:50.451708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:50.451708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:50.451708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:50.451708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:50.451708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:50.451708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:50.451708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:50.451708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:50.451708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:50.451708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:01:50.451708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:01:50.451708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:01:50.452981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:01:50.452981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:50.452981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:50.452981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:50.452981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:50.452981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:50.452981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:50.452981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:50.452981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:50.452981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:50.452981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:50.452981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:50.452981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:50.452981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:50.452981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:50.452981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:50.452981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:50.454213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:01:50.454213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:01:50.454213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:01:50.454213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:01:50.454213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:01:50.454213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:01:50.454213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:01:50.454213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 16:01:50.454213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 622/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 16:01:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:01:50.459942 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5183f-c4c2-4d18-b675-935b2d11b8d5 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:50.587332 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5183f-c4c2-4d18-b675-935b2d11b8d5 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:50.625242 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e39f18-fe62-4c90-af0a-1c38b75acc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:50.625883 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e39f18-fe62-4c90-af0a-1c38b75acc4c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:50.626247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e39f18-fe62-4c90-af0a-1c38b75acc4c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:50.626535 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e39f18-fe62-4c90-af0a-1c38b75acc4c None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:50.627316 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e39f18-fe62-4c90-af0a-1c38b75acc4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:50.627627 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e39f18-fe62-4c90-af0a-1c38b75acc4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:50.629057 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e39f18-fe62-4c90-af0a-1c38b75acc4c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:50.642574 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 623/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 16:01:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:50.649188 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31fdf2-93f8-4ef2-87d4-50c37975de54 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:50.764234 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31fdf2-93f8-4ef2-87d4-50c37975de54 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:50.803204 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d63fd3d-b534-49f8-bf68-01b2411bd79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:50.803885 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63fd3d-b534-49f8-bf68-01b2411bd79f None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:50.804148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63fd3d-b534-49f8-bf68-01b2411bd79f None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:50.804451 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63fd3d-b534-49f8-bf68-01b2411bd79f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:50.806602 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d63fd3d-b534-49f8-bf68-01b2411bd79f None admin] RBAC: Adding query filter params (role.id=cfc67444e9944ffc87173be8c9245108, scope.project.id=b1177a85a0de4276898daf4a24ff069f, user.id=36b63614297346e29ff276eab444fe5b) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:50.806937 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d63fd3d-b534-49f8-bf68-01b2411bd79f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:50.808451 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d63fd3d-b534-49f8-bf68-01b2411bd79f None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:50.815322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 623/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 16:01:50 2022] GET /identity/v3/role_assignments?user.id=36b63614297346e29ff276eab444fe5b&scope.project.id=b1177a85a0de4276898daf4a24ff069f&role.id=cfc67444e9944ffc87173be8c9245108 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:53.173386 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dedae0-923c-4d3a-894c-12b37542e544 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:53.173963 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dedae0-923c-4d3a-894c-12b37542e544 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:53.174084 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dedae0-923c-4d3a-894c-12b37542e544 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:53.175615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 624/1247] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:01:53.181590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc440c-6333-4f7d-b409-9dc29c5ba5e1 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:53.182110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc440c-6333-4f7d-b409-9dc29c5ba5e1 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:53.182346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc440c-6333-4f7d-b409-9dc29c5ba5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:53.187916 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:53.201797 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-3dfc440c-6333-4f7d-b409-9dc29c5ba5e1 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:01:53.203179 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-3dfc440c-6333-4f7d-b409-9dc29c5ba5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:01:53.287211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 624/1248] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:01:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:01:53.316222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dedae0-923c-4d3a-894c-12b37542e544 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:53.337829 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:53.445542 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dedae0-923c-4d3a-894c-12b37542e544 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:53.490816 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c8fdd-fac2-4a30-8c4c-e5c18279933a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:53.491623 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c8fdd-fac2-4a30-8c4c-e5c18279933a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:53.491914 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c8fdd-fac2-4a30-8c4c-e5c18279933a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:53.492906 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c8fdd-fac2-4a30-8c4c-e5c18279933a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:53.506089 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c8fdd-fac2-4a30-8c4c-e5c18279933a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:01:53.520019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-b08c8fdd-fac2-4a30-8c4c-e5c18279933a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:01:53.520019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:53.520019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:53.520019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:53.520019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:53.520019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:53.520019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:53.520019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:53.520019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:53.520019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:53.520019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:53.520019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:53.520019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:01:53.520019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:01:53.520019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:01:53.520019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:01:53.521420 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:01:53.521420 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:01:53.521420 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:01:53.521420 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:01:53.521420 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:53.521420 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:53.521420 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:53.521420 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:53.521420 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:53.521420 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:53.521420 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:53.521420 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:53.521420 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:53.521420 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:53.521420 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:53.521420 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:53.522549 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:53.522549 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:53.522549 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:53.522549 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:53.522549 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:01:53.522549 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:01:53.522549 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:01:53.522549 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:01:53.522549 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:01:53.522549 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:01:53.522549 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:01:53.522549 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 16:01:53.522549 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 625/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 16:01:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:01:53.528137 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc440c-6333-4f7d-b409-9dc29c5ba5e1 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:53.648047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc440c-6333-4f7d-b409-9dc29c5ba5e1 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:53.680714 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:53.686861 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135169a4-05c3-4ee9-855e-c34881a52a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:53.687657 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135169a4-05c3-4ee9-855e-c34881a52a88 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:53.687945 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135169a4-05c3-4ee9-855e-c34881a52a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:53.688254 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135169a4-05c3-4ee9-855e-c34881a52a88 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:53.689139 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135169a4-05c3-4ee9-855e-c34881a52a88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:53.689549 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135169a4-05c3-4ee9-855e-c34881a52a88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:53.691315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135169a4-05c3-4ee9-855e-c34881a52a88 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:53.701944 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 625/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 16:01:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:53.708022 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c8fdd-fac2-4a30-8c4c-e5c18279933a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:53.803860 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:53.832012 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c8fdd-fac2-4a30-8c4c-e5c18279933a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:53.875095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd497faa-4fd3-4eaa-b3b1-25550c0e8c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:53.875906 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd497faa-4fd3-4eaa-b3b1-25550c0e8c1c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:53.876196 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd497faa-4fd3-4eaa-b3b1-25550c0e8c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:53.876504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd497faa-4fd3-4eaa-b3b1-25550c0e8c1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:53.892782 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd497faa-4fd3-4eaa-b3b1-25550c0e8c1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:53.895091 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd497faa-4fd3-4eaa-b3b1-25550c0e8c1c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:53.919486 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-bd497faa-4fd3-4eaa-b3b1-25550c0e8c1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:01:53.919486 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:53.919486 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:53.919486 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:53.919486 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:53.919486 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:53.919486 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:53.919486 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:53.919486 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:53.919486 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:53.919486 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:53.919486 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:53.919486 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:01:53.919486 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:01:53.919486 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:01:53.920620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:01:53.920620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:53.920620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:53.920620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:53.920620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:53.920620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:53.920620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:53.920620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:53.920620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:53.920620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:53.920620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:53.920620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:53.920620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:53.920620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:53.920620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:53.920620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:53.920620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:53.921676 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:01:53.921676 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:01:53.921676 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:01:53.921676 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 16:01:53.921676 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 626/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 16:01:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:01:53.927649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135169a4-05c3-4ee9-855e-c34881a52a88 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:54.048345 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135169a4-05c3-4ee9-855e-c34881a52a88 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:54.088036 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bef3a2-285c-45ed-baa4-6d637e713f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:54.088776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bef3a2-285c-45ed-baa4-6d637e713f50 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:54.089089 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bef3a2-285c-45ed-baa4-6d637e713f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:54.089376 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bef3a2-285c-45ed-baa4-6d637e713f50 None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:54.090137 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bef3a2-285c-45ed-baa4-6d637e713f50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:54.090451 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bef3a2-285c-45ed-baa4-6d637e713f50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:54.091960 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bef3a2-285c-45ed-baa4-6d637e713f50 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:54.107586 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 626/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 16:01:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:01:54.113291 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd497faa-4fd3-4eaa-b3b1-25550c0e8c1c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:54.242569 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd497faa-4fd3-4eaa-b3b1-25550c0e8c1c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:54.278761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:54.290633 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd81c87-2490-4f25-9bc0-708608b9a46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:54.292516 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd81c87-2490-4f25-9bc0-708608b9a46c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:54.292940 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd81c87-2490-4f25-9bc0-708608b9a46c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:54.293220 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd81c87-2490-4f25-9bc0-708608b9a46c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:54.303291 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd81c87-2490-4f25-9bc0-708608b9a46c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:54.305502 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd81c87-2490-4f25-9bc0-708608b9a46c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:54.339120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-3cd81c87-2490-4f25-9bc0-708608b9a46c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:01:54.339120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:54.339120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:54.339120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:54.339120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:54.339120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:54.339120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:54.339120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:54.339120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:54.339120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:54.339120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:54.339120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:54.339120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:01:54.339120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:01:54.339120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:01:54.339120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:01:54.341363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:54.341363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:54.341363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:01:54.341363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:01:54.341363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:01:54.341363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:01:54.341363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:54.341363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:54.341363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:54.341363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:54.341363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:54.341363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:54.341363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:54.341363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:54.341363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:54.341363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:54.341363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:54.341363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:54.343872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:54.343872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:54.343872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:01:54.343872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:01:54.343872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:01:54.343872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:01:54.343872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:01:54.343872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:01:54.343872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:01:54.343872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 16:01:54.343872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 627/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 16:01:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:01:54.348373 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bef3a2-285c-45ed-baa4-6d637e713f50 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:54.407615 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:54.456084 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bef3a2-285c-45ed-baa4-6d637e713f50 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:54.498373 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8fc74e-7e9c-45c6-9406-c0fe0e474426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:54.499081 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fc74e-7e9c-45c6-9406-c0fe0e474426 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:54.499505 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fc74e-7e9c-45c6-9406-c0fe0e474426 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:54.499670 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fc74e-7e9c-45c6-9406-c0fe0e474426 None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:54.500535 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8fc74e-7e9c-45c6-9406-c0fe0e474426 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:54.500845 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8fc74e-7e9c-45c6-9406-c0fe0e474426 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:54.502561 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8fc74e-7e9c-45c6-9406-c0fe0e474426 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:54.526947 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 627/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jun 16 16:01:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:54.532780 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd81c87-2490-4f25-9bc0-708608b9a46c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:54.657134 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd81c87-2490-4f25-9bc0-708608b9a46c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:54.703356 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6c1cb-5198-44cd-aa19-bcf5e16161a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:54.704063 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6c1cb-5198-44cd-aa19-bcf5e16161a8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:54.704378 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6c1cb-5198-44cd-aa19-bcf5e16161a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:54.704658 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6c1cb-5198-44cd-aa19-bcf5e16161a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:54.766588 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:54.782488 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6c1cb-5198-44cd-aa19-bcf5e16161a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:54.783975 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6c1cb-5198-44cd-aa19-bcf5e16161a8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:54.899622 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 628/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 16:01:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:01:54.905904 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8fc74e-7e9c-45c6-9406-c0fe0e474426 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:55.018937 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8fc74e-7e9c-45c6-9406-c0fe0e474426 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:55.033757 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:55.058687 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189faee0-dd46-43ea-ab05-349bb12f73af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:55.059927 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189faee0-dd46-43ea-ab05-349bb12f73af None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:55.060326 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189faee0-dd46-43ea-ab05-349bb12f73af None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:55.060642 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189faee0-dd46-43ea-ab05-349bb12f73af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:55.072064 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189faee0-dd46-43ea-ab05-349bb12f73af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:55.073700 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189faee0-dd46-43ea-ab05-349bb12f73af None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:55.093706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-189faee0-dd46-43ea-ab05-349bb12f73af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:01:55.093706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:55.093706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:55.093706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:55.093706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:55.093706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:55.093706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:55.093706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:55.093706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:55.093706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:55.093706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:55.093706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:55.093706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:01:55.093706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:01:55.093706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:01:55.095147 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:01:55.095147 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:55.095147 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:55.095147 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:55.095147 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:55.095147 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:55.095147 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:55.095147 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:55.095147 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:55.095147 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:55.095147 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:55.095147 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:55.095147 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:55.095147 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:55.095147 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:55.095147 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:55.095147 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:55.096418 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:01:55.096418 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:01:55.096418 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:01:55.096418 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 16:01:55.096418 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 628/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 16:01:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:01:55.102031 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6c1cb-5198-44cd-aa19-bcf5e16161a8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:55.207841 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:55.222104 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6c1cb-5198-44cd-aa19-bcf5e16161a8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:55.264144 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1e439-36df-4547-a99e-3b5f09c081d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:55.265112 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1e439-36df-4547-a99e-3b5f09c081d4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:55.265112 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1e439-36df-4547-a99e-3b5f09c081d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:55.265336 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1e439-36df-4547-a99e-3b5f09c081d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:55.266253 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce1e439-36df-4547-a99e-3b5f09c081d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:55.266587 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce1e439-36df-4547-a99e-3b5f09c081d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:55.268680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce1e439-36df-4547-a99e-3b5f09c081d4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:55.283635 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 629/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 16:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:55.289698 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189faee0-dd46-43ea-ab05-349bb12f73af None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:55.402937 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189faee0-dd46-43ea-ab05-349bb12f73af None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:55.441203 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2090ab-72bb-4af5-a59f-60e23801a0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:55.442108 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2090ab-72bb-4af5-a59f-60e23801a0ae None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:55.442347 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2090ab-72bb-4af5-a59f-60e23801a0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:55.442621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2090ab-72bb-4af5-a59f-60e23801a0ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:55.451014 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2090ab-72bb-4af5-a59f-60e23801a0ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:55.452544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2090ab-72bb-4af5-a59f-60e23801a0ae None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:55.470117 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-df2090ab-72bb-4af5-a59f-60e23801a0ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:01:55.470117 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:55.470117 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:55.470117 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:55.470117 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:55.470117 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:55.470117 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:55.470117 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:55.470117 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:55.470117 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:55.470117 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:55.470117 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:55.470117 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:01:55.470117 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:01:55.470117 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:01:55.471516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:01:55.471516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:55.471516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:55.471516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:55.471516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:55.471516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:55.471516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:55.471516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:55.471516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:55.471516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:55.471516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:55.471516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:55.471516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:55.471516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:55.471516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:55.471516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:55.471516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:55.472914 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:01:55.472914 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:01:55.472914 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:01:55.472914 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:01:55.472914 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:01:55.472914 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:01:55.472914 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:01:55.472914 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 16:01:55.472914 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 629/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 16:01:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:01:55.478930 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1e439-36df-4547-a99e-3b5f09c081d4 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:55.596989 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1e439-36df-4547-a99e-3b5f09c081d4 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:55.635578 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99f045-6589-43b6-ab8d-0d72cbb38c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:55.636253 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99f045-6589-43b6-ab8d-0d72cbb38c6d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:55.636538 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99f045-6589-43b6-ab8d-0d72cbb38c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:55.636817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99f045-6589-43b6-ab8d-0d72cbb38c6d None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:55.637616 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c99f045-6589-43b6-ab8d-0d72cbb38c6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:55.637949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c99f045-6589-43b6-ab8d-0d72cbb38c6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:55.639423 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c99f045-6589-43b6-ab8d-0d72cbb38c6d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:55.653527 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 630/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 16:01:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:55.659936 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2090ab-72bb-4af5-a59f-60e23801a0ae None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:55.689539 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:55.777615 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2090ab-72bb-4af5-a59f-60e23801a0ae None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:55.816484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4200f-831a-4ebb-ab19-707538f427c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:55.817150 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4200f-831a-4ebb-ab19-707538f427c5 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:55.817430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4200f-831a-4ebb-ab19-707538f427c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:55.817705 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4200f-831a-4ebb-ab19-707538f427c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:55.818566 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada4200f-831a-4ebb-ab19-707538f427c5 None admin] RBAC: Adding query filter params (role.id=bbbc77ec645d4ff5a8bf29ce0377f578, scope.project.id=b1177a85a0de4276898daf4a24ff069f, user.id=36b63614297346e29ff276eab444fe5b) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:55.818878 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada4200f-831a-4ebb-ab19-707538f427c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:55.820393 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada4200f-831a-4ebb-ab19-707538f427c5 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:55.826352 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 630/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 16:01:55 2022] GET /identity/v3/role_assignments?user.id=36b63614297346e29ff276eab444fe5b&scope.project.id=b1177a85a0de4276898daf4a24ff069f&role.id=bbbc77ec645d4ff5a8bf29ce0377f578 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:01:57.986444 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5e186-fdd1-4e31-9f86-c2a1937af09d None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:57.987049 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5e186-fdd1-4e31-9f86-c2a1937af09d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:57.987049 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5e186-fdd1-4e31-9f86-c2a1937af09d None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:57.988386 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 631/1261] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:01:57.994148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5ee7e-c6af-4042-a2ab-3b355b70f3c8 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:57.994686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5ee7e-c6af-4042-a2ab-3b355b70f3c8 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:57.994803 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5ee7e-c6af-4042-a2ab-3b355b70f3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:58.013083 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-50b5ee7e-c6af-4042-a2ab-3b355b70f3c8 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:01:58.014346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-50b5ee7e-c6af-4042-a2ab-3b355b70f3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:01:58.091181 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 631/1262] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:01:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:01:58.114077 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c5e186-fdd1-4e31-9f86-c2a1937af09d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:58.126403 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:58.223456 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c5e186-fdd1-4e31-9f86-c2a1937af09d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:58.260555 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a046b89-8638-4546-9aba-582384ed7907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:58.261343 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a046b89-8638-4546-9aba-582384ed7907 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:58.261623 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a046b89-8638-4546-9aba-582384ed7907 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:58.261898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a046b89-8638-4546-9aba-582384ed7907 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:58.272441 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a046b89-8638-4546-9aba-582384ed7907 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:01:58.280147 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-2a046b89-8638-4546-9aba-582384ed7907 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:01:58.280147 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:58.280147 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:58.280147 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:58.280147 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:58.280147 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:58.280147 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:58.280147 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:58.280147 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:58.280147 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:58.280147 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:58.280147 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:58.280147 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:01:58.280147 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:01:58.280147 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:01:58.280147 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:01:58.281405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:01:58.281405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:01:58.281405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:01:58.281405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:01:58.281405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:58.281405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:58.281405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:58.281405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:58.281405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:58.281405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:58.281405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:58.281405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:58.281405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:58.281405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:58.281405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:58.281405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:58.282328 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:58.282328 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:58.282328 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:58.282328 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:58.282328 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:01:58.282328 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:01:58.282328 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:01:58.282328 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:01:58.282328 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:01:58.282328 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:01:58.282328 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:01:58.282328 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 16:01:58.282328 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 632/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 16:01:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:01:58.287758 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5ee7e-c6af-4042-a2ab-3b355b70f3c8 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:58.308108 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:58.394338 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5ee7e-c6af-4042-a2ab-3b355b70f3c8 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:58.428190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cc600-c516-4943-83ba-9d6594eafcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:58.428854 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cc600-c516-4943-83ba-9d6594eafcd2 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:58.429139 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cc600-c516-4943-83ba-9d6594eafcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:58.430128 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cc600-c516-4943-83ba-9d6594eafcd2 None admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:58.431078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1cc600-c516-4943-83ba-9d6594eafcd2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:58.431388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1cc600-c516-4943-83ba-9d6594eafcd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:58.432791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1cc600-c516-4943-83ba-9d6594eafcd2 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:58.440886 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 632/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 16:01:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:58.446643 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a046b89-8638-4546-9aba-582384ed7907 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:58.533404 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:58.570831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a046b89-8638-4546-9aba-582384ed7907 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:58.604255 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8e9ec0-d3f0-4159-8694-15b061e2c2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:58.605054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e9ec0-d3f0-4159-8694-15b061e2c2de None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:58.605337 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e9ec0-d3f0-4159-8694-15b061e2c2de None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:58.605612 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e9ec0-d3f0-4159-8694-15b061e2c2de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:58.616552 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8e9ec0-d3f0-4159-8694-15b061e2c2de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:58.618746 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8e9ec0-d3f0-4159-8694-15b061e2c2de None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:58.635831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-5b8e9ec0-d3f0-4159-8694-15b061e2c2de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:01:58.635831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:58.635831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:58.635831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:58.635831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:58.635831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:58.635831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:58.635831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:58.635831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:58.635831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:58.635831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:58.635831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:58.635831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:01:58.635831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:01:58.635831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:01:58.636898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:01:58.636898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:58.636898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:58.636898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:58.636898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:58.636898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:58.636898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:58.636898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:58.636898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:58.636898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:58.636898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:58.636898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:58.636898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:58.636898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:58.636898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:58.636898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:58.636898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:58.637879 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:01:58.637879 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:01:58.637879 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:01:58.637879 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 16:01:58.637879 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 633/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 16:01:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:01:58.642801 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cc600-c516-4943-83ba-9d6594eafcd2 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:58.729574 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:58.748279 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cc600-c516-4943-83ba-9d6594eafcd2 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:58.781553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e973aba-6617-4117-bbe0-5dbdbd32db8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:58.782249 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e973aba-6617-4117-bbe0-5dbdbd32db8e None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:58.782532 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e973aba-6617-4117-bbe0-5dbdbd32db8e None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:58.782809 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e973aba-6617-4117-bbe0-5dbdbd32db8e None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:58.784074 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e973aba-6617-4117-bbe0-5dbdbd32db8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:58.784074 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e973aba-6617-4117-bbe0-5dbdbd32db8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:58.785215 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e973aba-6617-4117-bbe0-5dbdbd32db8e None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:58.798148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 633/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 16:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:58.803810 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8e9ec0-d3f0-4159-8694-15b061e2c2de None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:58.902808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8e9ec0-d3f0-4159-8694-15b061e2c2de None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:58.922480 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:58.936645 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c6f12-99d0-4a2a-be99-87884e4d83e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:58.937446 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c6f12-99d0-4a2a-be99-87884e4d83e9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:58.937732 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c6f12-99d0-4a2a-be99-87884e4d83e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:58.938052 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c6f12-99d0-4a2a-be99-87884e4d83e9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:58.946440 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c6f12-99d0-4a2a-be99-87884e4d83e9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:58.947984 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c6f12-99d0-4a2a-be99-87884e4d83e9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:58.964560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-602c6f12-99d0-4a2a-be99-87884e4d83e9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:01:58.964560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:58.964560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:58.964560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:58.964560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:58.964560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:58.964560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:58.964560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:58.964560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:58.964560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:58.964560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:58.964560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:58.964560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:01:58.964560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:01:58.964560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:01:58.964560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:01:58.965815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:58.965815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:58.965815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:01:58.965815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:01:58.965815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:01:58.965815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:01:58.965815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:58.965815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:58.965815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:58.965815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:58.965815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:58.965815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:58.965815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:58.965815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:58.965815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:58.965815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:58.965815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:58.965815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:58.967209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:58.967209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:58.967209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:01:58.967209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:01:58.967209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:01:58.967209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:01:58.967209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:01:58.967209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:01:58.967209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:01:58.967209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 16:01:58.967209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 634/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 16:01:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:01:58.971751 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e973aba-6617-4117-bbe0-5dbdbd32db8e None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:59.076274 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e973aba-6617-4117-bbe0-5dbdbd32db8e None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:59.112093 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73d014-1674-4863-a27b-64a313963444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:59.112770 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73d014-1674-4863-a27b-64a313963444 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:59.113056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73d014-1674-4863-a27b-64a313963444 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:59.113333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73d014-1674-4863-a27b-64a313963444 None admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:59.114199 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73d014-1674-4863-a27b-64a313963444 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:59.114511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73d014-1674-4863-a27b-64a313963444 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:59.115977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73d014-1674-4863-a27b-64a313963444 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:59.125039 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:59.134770 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 634/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jun 16 16:01:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:59.140513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c6f12-99d0-4a2a-be99-87884e4d83e9 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:59.248819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c6f12-99d0-4a2a-be99-87884e4d83e9 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:59.284656 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8911634-e618-4abf-aee9-2e1a3eec383d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:59.285447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8911634-e618-4abf-aee9-2e1a3eec383d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:59.285732 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8911634-e618-4abf-aee9-2e1a3eec383d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:59.286036 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8911634-e618-4abf-aee9-2e1a3eec383d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:59.297772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8911634-e618-4abf-aee9-2e1a3eec383d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:59.299276 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8911634-e618-4abf-aee9-2e1a3eec383d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:59.316987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-e8911634-e618-4abf-aee9-2e1a3eec383d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:01:59.316987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:59.316987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:59.316987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:59.316987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:59.316987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:59.316987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:59.316987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:59.316987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:59.316987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:59.316987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:59.316987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:59.316987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:01:59.316987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:01:59.316987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:01:59.318120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:01:59.318120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:59.318120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:59.318120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:59.318120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:59.318120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:59.318120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:59.318120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:59.318120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:59.318120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:59.318120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:59.318120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:59.318120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:59.318120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:59.318120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:59.318120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:59.318120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:59.319437 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:01:59.319437 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:01:59.319437 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:01:59.319437 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 16:01:59.319437 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 635/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 16:01:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:01:59.324044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73d014-1674-4863-a27b-64a313963444 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:59.434003 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73d014-1674-4863-a27b-64a313963444 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:59.481326 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a06e8-f36e-41b4-9698-bc49408828c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:59.482170 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a06e8-f36e-41b4-9698-bc49408828c0 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:59.482498 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a06e8-f36e-41b4-9698-bc49408828c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:59.482905 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a06e8-f36e-41b4-9698-bc49408828c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:59.483804 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a06e8-f36e-41b4-9698-bc49408828c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:59.484198 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a06e8-f36e-41b4-9698-bc49408828c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:59.486259 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a06e8-f36e-41b4-9698-bc49408828c0 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:59.500914 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 635/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 16:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:01:59.506750 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8911634-e618-4abf-aee9-2e1a3eec383d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:59.542947 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:59.619921 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8911634-e618-4abf-aee9-2e1a3eec383d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:59.655184 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34873fe8-ded7-4509-a2de-795501e6ddde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:59.656079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34873fe8-ded7-4509-a2de-795501e6ddde None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:59.656426 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34873fe8-ded7-4509-a2de-795501e6ddde None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:59.656743 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34873fe8-ded7-4509-a2de-795501e6ddde None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:59.664267 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34873fe8-ded7-4509-a2de-795501e6ddde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:59.665795 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34873fe8-ded7-4509-a2de-795501e6ddde None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:59.681274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-34873fe8-ded7-4509-a2de-795501e6ddde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:01:59.681274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:59.681274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:59.681274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:59.681274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:59.681274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:59.681274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:59.681274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:59.681274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:59.681274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:59.681274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:59.681274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:59.681274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:01:59.681274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:01:59.681274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:01:59.682534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:01:59.682534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:59.682534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:59.682534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:59.682534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:59.682534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:59.682534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:59.682534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:59.682534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:59.682534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:59.682534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:59.682534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:59.682534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:59.682534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:59.682534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:59.682534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:59.682534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:59.683807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:01:59.683807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:01:59.683807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:01:59.683807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:01:59.683807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:01:59.683807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:01:59.683807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:01:59.683807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 16:01:59.683807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 636/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 16:01:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:01:59.689201 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a06e8-f36e-41b4-9698-bc49408828c0 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:59.755062 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:59.799344 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a06e8-f36e-41b4-9698-bc49408828c0 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:59.833213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857660df-2da9-4445-9edb-f168725ff230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:59.833873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857660df-2da9-4445-9edb-f168725ff230 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:59.834187 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857660df-2da9-4445-9edb-f168725ff230 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:59.834463 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857660df-2da9-4445-9edb-f168725ff230 None admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:59.835253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857660df-2da9-4445-9edb-f168725ff230 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:59.835563 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857660df-2da9-4445-9edb-f168725ff230 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:59.837556 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857660df-2da9-4445-9edb-f168725ff230 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:59.849810 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 636/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 16:01:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:01:59.856081 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34873fe8-ded7-4509-a2de-795501e6ddde None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:59.949567 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:59.959031 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34873fe8-ded7-4509-a2de-795501e6ddde None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:59.994620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92737f2-4b49-4318-bdaa-3632efc49070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:59.995636 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92737f2-4b49-4318-bdaa-3632efc49070 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:59.995919 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92737f2-4b49-4318-bdaa-3632efc49070 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:59.996213 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92737f2-4b49-4318-bdaa-3632efc49070 None admin] PATH_INFO: `/v3/projects/b1177a85a0de4276898daf4a24ff069f/users/36b63614297346e29ff276eab444fe5b/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:59.998362 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92737f2-4b49-4318-bdaa-3632efc49070 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1177a85a0de4276898daf4a24ff069f, user_id=36b63614297346e29ff276eab444fe5b, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:59.999873 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92737f2-4b49-4318-bdaa-3632efc49070 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:00.016134 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 637/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 16:01:59 2022] PUT /identity/v3/projects/b1177a85a0de4276898daf4a24ff069f/users/36b63614297346e29ff276eab444fe5b/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:02:02.165505 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104bac6-c19f-4985-92dd-b2f78cfb96b8 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:02.166108 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104bac6-c19f-4985-92dd-b2f78cfb96b8 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:02.166108 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104bac6-c19f-4985-92dd-b2f78cfb96b8 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:02.167477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 637/1274] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:02:02.172871 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a90a8-b4aa-409d-b15a-47a5818987ff None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:02.173302 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a90a8-b4aa-409d-b15a-47a5818987ff None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:02.173435 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a90a8-b4aa-409d-b15a-47a5818987ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:02.190896 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b65a90a8-b4aa-409d-b15a-47a5818987ff None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:02:02.192100 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b65a90a8-b4aa-409d-b15a-47a5818987ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:02:02.262837 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 638/1275] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:02:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:02:02.286119 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104bac6-c19f-4985-92dd-b2f78cfb96b8 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:02.389486 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104bac6-c19f-4985-92dd-b2f78cfb96b8 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:02.407684 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:02.425784 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f732374-fecc-41be-a5e0-3b901c6ef641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:02.426536 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f732374-fecc-41be-a5e0-3b901c6ef641 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:02.426824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f732374-fecc-41be-a5e0-3b901c6ef641 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:02.427100 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f732374-fecc-41be-a5e0-3b901c6ef641 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:02.437755 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f732374-fecc-41be-a5e0-3b901c6ef641 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:02:02.445469 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-2f732374-fecc-41be-a5e0-3b901c6ef641 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:02:02.445469 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:02.445469 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:02.445469 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:02.445469 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:02.445469 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:02.445469 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:02.445469 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:02.445469 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:02.445469 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:02.445469 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:02.445469 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:02.445469 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:02:02.445469 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:02:02.445469 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:02:02.445469 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:02:02.446607 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:02:02.446607 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:02:02.446607 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:02:02.446607 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:02:02.446607 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:02.446607 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:02.446607 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:02.446607 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:02.446607 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:02.446607 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:02.446607 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:02.446607 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:02.446607 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:02.446607 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:02.446607 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:02.446607 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:02.447651 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:02.447651 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:02.447651 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:02.447651 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:02.447651 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:02:02.447651 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:02:02.447651 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:02:02.447651 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:02:02.447651 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:02:02.447651 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:02:02.447651 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:02:02.447651 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 16:02:02.447651 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 638/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 16:02:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:02.453128 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a90a8-b4aa-409d-b15a-47a5818987ff None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:02.555780 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:02.574656 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a90a8-b4aa-409d-b15a-47a5818987ff None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:02.608905 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b21c9-7127-4825-9bfa-566b7cbd62e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:02.609579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b21c9-7127-4825-9bfa-566b7cbd62e3 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:02.609855 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b21c9-7127-4825-9bfa-566b7cbd62e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:02.610155 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b21c9-7127-4825-9bfa-566b7cbd62e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:02.610898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b21c9-7127-4825-9bfa-566b7cbd62e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:02.611195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b21c9-7127-4825-9bfa-566b7cbd62e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:02.612569 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b21c9-7127-4825-9bfa-566b7cbd62e3 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:02.621314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 639/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 16:02:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:02:02.627132 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f732374-fecc-41be-a5e0-3b901c6ef641 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:02.727261 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f732374-fecc-41be-a5e0-3b901c6ef641 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:02.761197 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e9543-603a-49af-b444-4fffe06d606a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:02.762029 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e9543-603a-49af-b444-4fffe06d606a None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:02.762318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e9543-603a-49af-b444-4fffe06d606a None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:02.762596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e9543-603a-49af-b444-4fffe06d606a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:02.772900 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e9543-603a-49af-b444-4fffe06d606a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:02.774417 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e9543-603a-49af-b444-4fffe06d606a None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:02.792452 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-a83e9543-603a-49af-b444-4fffe06d606a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:02.792452 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:02.792452 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:02.792452 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:02.792452 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:02.792452 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:02.792452 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:02.792452 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:02.792452 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:02.792452 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:02.792452 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:02.792452 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:02.792452 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:02:02.792452 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:02:02.792452 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:02:02.793658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:02:02.793658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:02.793658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:02.793658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:02.793658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:02.793658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:02.793658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:02.793658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:02.793658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:02.793658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:02.793658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:02.793658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:02.793658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:02.793658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:02.793658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:02.793658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:02.793658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:02.794968 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:02:02.794968 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:02:02.794968 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:02.794968 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 16:02:02.794968 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 639/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 16:02:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:02.799539 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b21c9-7127-4825-9bfa-566b7cbd62e3 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:02.897383 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b21c9-7127-4825-9bfa-566b7cbd62e3 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:02.932340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa7131-884a-4fb8-a5b1-6a20a084730c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:02.933000 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa7131-884a-4fb8-a5b1-6a20a084730c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:02.933285 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa7131-884a-4fb8-a5b1-6a20a084730c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:02.933561 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa7131-884a-4fb8-a5b1-6a20a084730c None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:02.934303 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa7131-884a-4fb8-a5b1-6a20a084730c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:02.934610 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa7131-884a-4fb8-a5b1-6a20a084730c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:02.935975 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa7131-884a-4fb8-a5b1-6a20a084730c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:02.947632 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 640/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 16:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:02.953268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e9543-603a-49af-b444-4fffe06d606a None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:02.984766 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:03.055041 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e9543-603a-49af-b444-4fffe06d606a None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:03.088987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2846272c-24d0-45a0-80b6-5f0113fbff33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:03.089786 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2846272c-24d0-45a0-80b6-5f0113fbff33 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:03.090101 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2846272c-24d0-45a0-80b6-5f0113fbff33 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:03.090380 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2846272c-24d0-45a0-80b6-5f0113fbff33 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:03.097865 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2846272c-24d0-45a0-80b6-5f0113fbff33 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:03.099356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2846272c-24d0-45a0-80b6-5f0113fbff33 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:03.114138 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-2846272c-24d0-45a0-80b6-5f0113fbff33 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:02:03.114138 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:03.114138 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:03.114138 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:03.114138 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:03.114138 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:03.114138 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:03.114138 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:03.114138 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:03.114138 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:03.114138 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:03.114138 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:03.114138 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:02:03.114138 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:02:03.114138 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:02:03.114138 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:02:03.115282 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:03.115282 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:03.115282 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:02:03.115282 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:02:03.115282 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:02:03.115282 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:02:03.115282 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:03.115282 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:03.115282 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:03.115282 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:03.115282 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:03.115282 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:03.115282 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:03.115282 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:03.115282 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:03.115282 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:03.115282 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:03.115282 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:03.116429 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:03.116429 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:03.116429 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:02:03.116429 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:02:03.116429 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:02:03.116429 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:02:03.116429 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:02:03.116429 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:02:03.116429 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:02:03.116429 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 16:02:03.116429 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 640/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 16:02:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:03.121668 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa7131-884a-4fb8-a5b1-6a20a084730c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:03.125413 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:03.221902 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa7131-884a-4fb8-a5b1-6a20a084730c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:03.256967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b537f0d-4f83-4bc5-ae38-06e043f325fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:03.258542 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b537f0d-4f83-4bc5-ae38-06e043f325fe None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:03.258906 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b537f0d-4f83-4bc5-ae38-06e043f325fe None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:03.259200 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b537f0d-4f83-4bc5-ae38-06e043f325fe None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:03.260077 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b537f0d-4f83-4bc5-ae38-06e043f325fe None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:03.260391 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b537f0d-4f83-4bc5-ae38-06e043f325fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:03.261837 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b537f0d-4f83-4bc5-ae38-06e043f325fe None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:03.281605 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 641/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jun 16 16:02:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:03.287532 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2846272c-24d0-45a0-80b6-5f0113fbff33 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:03.383830 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:03.393493 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2846272c-24d0-45a0-80b6-5f0113fbff33 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:03.427641 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd48c8-43c5-475a-b9b9-6357b6cb0113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:03.428285 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd48c8-43c5-475a-b9b9-6357b6cb0113 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:03.428566 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd48c8-43c5-475a-b9b9-6357b6cb0113 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:03.428841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd48c8-43c5-475a-b9b9-6357b6cb0113 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:03.502897 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cd48c8-43c5-475a-b9b9-6357b6cb0113 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:03.504359 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cd48c8-43c5-475a-b9b9-6357b6cb0113 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:03.595687 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:03.607676 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 641/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 16:02:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:02:03.613423 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b537f0d-4f83-4bc5-ae38-06e043f325fe None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:03.674326 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:03.714869 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b537f0d-4f83-4bc5-ae38-06e043f325fe None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:03.751233 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80872501-66d8-4593-97c8-3390c6699a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:03.752025 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80872501-66d8-4593-97c8-3390c6699a0b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:03.752311 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80872501-66d8-4593-97c8-3390c6699a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:03.752590 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80872501-66d8-4593-97c8-3390c6699a0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:03.763935 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80872501-66d8-4593-97c8-3390c6699a0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:03.765393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80872501-66d8-4593-97c8-3390c6699a0b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:03.783151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-80872501-66d8-4593-97c8-3390c6699a0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:03.783151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:03.783151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:03.783151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:03.783151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:03.783151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:03.783151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:03.783151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:03.783151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:03.783151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:03.783151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:03.783151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:03.783151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:02:03.783151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:02:03.783151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:02:03.784138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:02:03.784138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:03.784138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:03.784138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:03.784138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:03.784138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:03.784138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:03.784138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:03.784138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:03.784138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:03.784138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:03.784138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:03.784138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:03.784138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:03.784138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:03.784138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:03.784138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:03.785061 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:02:03.785061 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:02:03.785061 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:03.785061 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 16:02:03.785061 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 642/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 16:02:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:03.790532 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd48c8-43c5-475a-b9b9-6357b6cb0113 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:03.895161 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd48c8-43c5-475a-b9b9-6357b6cb0113 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:03.930273 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adfc351-d4c7-4a8a-8075-9029f1674eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:03.931248 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adfc351-d4c7-4a8a-8075-9029f1674eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:03.931248 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adfc351-d4c7-4a8a-8075-9029f1674eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:03.931487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adfc351-d4c7-4a8a-8075-9029f1674eb1 None admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:03.932398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adfc351-d4c7-4a8a-8075-9029f1674eb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:03.932505 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adfc351-d4c7-4a8a-8075-9029f1674eb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:03.933898 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adfc351-d4c7-4a8a-8075-9029f1674eb1 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:03.945652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 642/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 16:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:03.951638 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80872501-66d8-4593-97c8-3390c6699a0b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:04.057809 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80872501-66d8-4593-97c8-3390c6699a0b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:04.072597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:04.094687 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd0e9f3-bb62-4a69-9802-27435715fa2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:04.095495 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0e9f3-bb62-4a69-9802-27435715fa2b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:04.095779 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0e9f3-bb62-4a69-9802-27435715fa2b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:04.096056 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0e9f3-bb62-4a69-9802-27435715fa2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:04.104317 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd0e9f3-bb62-4a69-9802-27435715fa2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:04.105772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd0e9f3-bb62-4a69-9802-27435715fa2b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:04.121354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-abd0e9f3-bb62-4a69-9802-27435715fa2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:02:04.121354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:04.121354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:04.121354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:04.121354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:04.121354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:04.121354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:04.121354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:04.121354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:04.121354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:04.121354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:04.121354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:04.121354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:02:04.121354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:02:04.121354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:02:04.127305 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:02:04.127305 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:04.127305 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:04.127305 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:04.127305 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:04.127305 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:04.127305 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:04.127305 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:04.127305 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:04.127305 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:04.127305 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:04.127305 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:04.127305 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:04.127305 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:04.127305 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:04.127305 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:04.127305 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:04.128495 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:02:04.128495 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:02:04.128495 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:02:04.128495 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:02:04.128495 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:02:04.128495 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:02:04.128495 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:02:04.128495 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jun 16 16:02:04.128495 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 643/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 16:02:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:04.129792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adfc351-d4c7-4a8a-8075-9029f1674eb1 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:04.187613 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:04.239923 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adfc351-d4c7-4a8a-8075-9029f1674eb1 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:04.275224 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c9ac3-f0b5-485b-9c60-d49a50a8c57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:04.275885 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c9ac3-f0b5-485b-9c60-d49a50a8c57a None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:04.276173 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c9ac3-f0b5-485b-9c60-d49a50a8c57a None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:04.276454 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c9ac3-f0b5-485b-9c60-d49a50a8c57a None admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:04.277244 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c9ac3-f0b5-485b-9c60-d49a50a8c57a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:04.277558 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c9ac3-f0b5-485b-9c60-d49a50a8c57a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:04.279050 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c9ac3-f0b5-485b-9c60-d49a50a8c57a None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:04.292914 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 643/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 16:02:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:04.299235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd0e9f3-bb62-4a69-9802-27435715fa2b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:04.405207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd0e9f3-bb62-4a69-9802-27435715fa2b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:04.441699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c15db-3f03-42d5-b3af-f896047600d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:04.442318 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c15db-3f03-42d5-b3af-f896047600d5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:04.442603 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c15db-3f03-42d5-b3af-f896047600d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:04.442881 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c15db-3f03-42d5-b3af-f896047600d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:04.443727 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c15db-3f03-42d5-b3af-f896047600d5 None admin] RBAC: Adding query filter params (role.id=bbbc77ec645d4ff5a8bf29ce0377f578, scope.project.id=b1177a85a0de4276898daf4a24ff069f, user.id=36b63614297346e29ff276eab444fe5b) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:04.444043 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c15db-3f03-42d5-b3af-f896047600d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:04.445648 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c15db-3f03-42d5-b3af-f896047600d5 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:04.452482 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 644/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 16:02:04 2022] GET /identity/v3/role_assignments?user.id=36b63614297346e29ff276eab444fe5b&scope.project.id=b1177a85a0de4276898daf4a24ff069f&role.id=bbbc77ec645d4ff5a8bf29ce0377f578 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:06.689661 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd953bf-a2f8-4566-9cc1-c95c1fb7201d None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:06.690300 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd953bf-a2f8-4566-9cc1-c95c1fb7201d None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:06.690300 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd953bf-a2f8-4566-9cc1-c95c1fb7201d None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:06.691659 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 644/1288] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:02:06.697298 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ce0df-98b7-4412-acf9-0dfceb4cbba4 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:06.697831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ce0df-98b7-4412-acf9-0dfceb4cbba4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:06.697831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ce0df-98b7-4412-acf9-0dfceb4cbba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:06.716202 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-118ce0df-98b7-4412-acf9-0dfceb4cbba4 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:02:06.717424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-118ce0df-98b7-4412-acf9-0dfceb4cbba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:02:06.749769 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:06.795152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 645/1289] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:02:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:02:06.817692 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd953bf-a2f8-4566-9cc1-c95c1fb7201d None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:06.924672 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd953bf-a2f8-4566-9cc1-c95c1fb7201d None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:06.936473 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:06.962308 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87805e-0f81-4e8f-ba1d-2981f512a369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:06.963028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87805e-0f81-4e8f-ba1d-2981f512a369 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:06.963309 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87805e-0f81-4e8f-ba1d-2981f512a369 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:06.963581 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87805e-0f81-4e8f-ba1d-2981f512a369 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:06.964243 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87805e-0f81-4e8f-ba1d-2981f512a369 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:06.965641 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87805e-0f81-4e8f-ba1d-2981f512a369 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:06.978807 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-8c87805e-0f81-4e8f-ba1d-2981f512a369 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 16:02:06.978807 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:06.978807 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:02:06.978807 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:02:06.978807 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:02:06.978807 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:02:06.978807 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:02:06.978807 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:02:06.978807 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 16:02:06.978807 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 16:02:06.978807 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:02:06.978807 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 16:02:06.978807 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:06.978807 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:06.978807 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:06.978807 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:06.978807 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:06.980195 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:06.980195 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:06.980195 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:06.980195 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:06.980195 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:06.980195 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:06.980195 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:02:06.980195 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:02:06.980195 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:02:06.980195 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:02:06.980195 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:06.980195 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:06.980195 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:06.980195 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:06.980195 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:06.980195 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:06.981044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:06.981044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:06.981044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:06.981044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:06.981044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:06.981044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:06.981044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:06.981044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:06.981044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:06.981044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:06.981044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:02:06.981044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:02:06.981044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 16:02:06.981044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 16:02:06.981044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 645/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jun 16 16:02:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:06.986136 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ce0df-98b7-4412-acf9-0dfceb4cbba4 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:07.089099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ce0df-98b7-4412-acf9-0dfceb4cbba4 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:07.123020 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e80ea-e5d9-4b66-b6a9-e569632cbcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:07.123691 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e80ea-e5d9-4b66-b6a9-e569632cbcc4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:07.123977 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e80ea-e5d9-4b66-b6a9-e569632cbcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:07.124252 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e80ea-e5d9-4b66-b6a9-e569632cbcc4 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:07.124968 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74e80ea-e5d9-4b66-b6a9-e569632cbcc4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:07.125275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74e80ea-e5d9-4b66-b6a9-e569632cbcc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:07.126684 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74e80ea-e5d9-4b66-b6a9-e569632cbcc4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:07.131756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 646/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jun 16 16:02:06 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:07.138856 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87805e-0f81-4e8f-ba1d-2981f512a369 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:07.243307 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87805e-0f81-4e8f-ba1d-2981f512a369 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:07.280017 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8a350-5e97-40c7-99ac-8f912b024f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:07.280685 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8a350-5e97-40c7-99ac-8f912b024f37 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:07.280970 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8a350-5e97-40c7-99ac-8f912b024f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:07.281246 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8a350-5e97-40c7-99ac-8f912b024f37 None admin] PATH_INFO: `/v3/services` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:07.281984 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d8a350-5e97-40c7-99ac-8f912b024f37 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:07.282296 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d8a350-5e97-40c7-99ac-8f912b024f37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:07.283681 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d8a350-5e97-40c7-99ac-8f912b024f37 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:07.288508 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 646/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jun 16 16:02:07 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:09.469431 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68371f12-2fbe-40a2-b346-5e7c74437a2d None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:09.470270 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68371f12-2fbe-40a2-b346-5e7c74437a2d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:09.470270 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68371f12-2fbe-40a2-b346-5e7c74437a2d None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:09.471547 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 647/1293] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:02:09.478611 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e9691-d637-4b48-92f7-fe15a59a2f0f None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:09.479289 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e9691-d637-4b48-92f7-fe15a59a2f0f None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:09.479289 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e9691-d637-4b48-92f7-fe15a59a2f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:09.501918 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-ec5e9691-d637-4b48-92f7-fe15a59a2f0f None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:02:09.503312 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-ec5e9691-d637-4b48-92f7-fe15a59a2f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:02:09.543226 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:09.582484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 647/1294] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:02:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:02:09.606892 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68371f12-2fbe-40a2-b346-5e7c74437a2d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:09.640250 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:09.718978 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68371f12-2fbe-40a2-b346-5e7c74437a2d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:09.755340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f3996-d377-496d-9b40-3ba7a57f9703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:09.755918 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f3996-d377-496d-9b40-3ba7a57f9703 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:09.756196 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f3996-d377-496d-9b40-3ba7a57f9703 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:09.756470 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f3996-d377-496d-9b40-3ba7a57f9703 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:09.757820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f3996-d377-496d-9b40-3ba7a57f9703 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:09.759937 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f3996-d377-496d-9b40-3ba7a57f9703 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:09.775788 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 648/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jun 16 16:02:09 2022] POST /identity/v3/services => generated 246 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:02:12.038857 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b331ace-0893-4c6f-9b20-e6b13c44acd1 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:12.039397 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b331ace-0893-4c6f-9b20-e6b13c44acd1 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:12.039397 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b331ace-0893-4c6f-9b20-e6b13c44acd1 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:12.040826 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 648/1296] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:02:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:02:12.046795 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a1209-192a-43f1-8b9d-bbfc983d0ffb None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:12.047402 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a1209-192a-43f1-8b9d-bbfc983d0ffb None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:12.047402 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a1209-192a-43f1-8b9d-bbfc983d0ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:12.065607 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d85a1209-192a-43f1-8b9d-bbfc983d0ffb None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:02:12.066944 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d85a1209-192a-43f1-8b9d-bbfc983d0ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:02:12.150926 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 649/1297] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:02:12 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:02:12.174957 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b331ace-0893-4c6f-9b20-e6b13c44acd1 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:12.282496 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b331ace-0893-4c6f-9b20-e6b13c44acd1 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:12.319533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ad1c6-d410-43c8-8f97-2be012c13f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:12.320513 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ad1c6-d410-43c8-8f97-2be012c13f29 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:12.320795 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ad1c6-d410-43c8-8f97-2be012c13f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:12.321073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ad1c6-d410-43c8-8f97-2be012c13f29 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:12.321739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2ad1c6-d410-43c8-8f97-2be012c13f29 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:12.323175 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2ad1c6-d410-43c8-8f97-2be012c13f29 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:12.337166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-7d2ad1c6-d410-43c8-8f97-2be012c13f29 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 16:02:12.337166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:12.337166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:02:12.337166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:02:12.337166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:02:12.337166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:02:12.337166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:02:12.337166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:02:12.337166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 16:02:12.337166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 16:02:12.337166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:02:12.337166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 16:02:12.337166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:12.337166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:12.337166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:12.337166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:12.337166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:12.338435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:12.338435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:12.338435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:12.338435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:12.338435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:12.338435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:12.338435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:02:12.338435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:02:12.338435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:02:12.338435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:02:12.338435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:12.338435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:12.338435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:12.338435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:12.338435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:12.338435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:12.339326 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:12.339326 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:12.339326 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:12.339326 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:12.339326 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:12.339326 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:12.339326 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:12.339326 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:12.339326 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:12.339326 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:12.339326 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:02:12.339326 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:02:12.339326 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 16:02:12.339326 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 16:02:12.339326 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 649/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jun 16 16:02:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:12.345025 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a1209-192a-43f1-8b9d-bbfc983d0ffb None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:12.365043 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:12.458467 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a1209-192a-43f1-8b9d-bbfc983d0ffb None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:12.503428 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3797ce8-641f-40b8-9dbd-5110a3de0da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:12.504207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3797ce8-641f-40b8-9dbd-5110a3de0da5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:12.504619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3797ce8-641f-40b8-9dbd-5110a3de0da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:12.505004 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3797ce8-641f-40b8-9dbd-5110a3de0da5 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:12.506571 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3797ce8-641f-40b8-9dbd-5110a3de0da5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:12.507139 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3797ce8-641f-40b8-9dbd-5110a3de0da5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:12.510047 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3797ce8-641f-40b8-9dbd-5110a3de0da5 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:12.520579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 650/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jun 16 16:02:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:12.527537 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ad1c6-d410-43c8-8f97-2be012c13f29 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:12.561774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:12.634859 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ad1c6-d410-43c8-8f97-2be012c13f29 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:12.669372 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf09862-9a09-4bdc-bf16-afc64aad350b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:12.670066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf09862-9a09-4bdc-bf16-afc64aad350b None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:12.670353 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf09862-9a09-4bdc-bf16-afc64aad350b None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:12.670629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf09862-9a09-4bdc-bf16-afc64aad350b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:12.671419 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf09862-9a09-4bdc-bf16-afc64aad350b None admin] RBAC: Adding query filter params (interface=public, service_id=12a63434dfad468694e111ba44f82c97, region_id=RegionOne) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:12.671734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf09862-9a09-4bdc-bf16-afc64aad350b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:12.673122 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf09862-9a09-4bdc-bf16-afc64aad350b None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:12.678656 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 650/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jun 16 16:02:12 2022] GET /identity/v3/endpoints?service_id=12a63434dfad468694e111ba44f82c97&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:12.684768 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3797ce8-641f-40b8-9dbd-5110a3de0da5 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:12.776482 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:12.799741 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3797ce8-641f-40b8-9dbd-5110a3de0da5 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:12.835640 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f6c33c-b457-4726-8e35-17e698236d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:12.836329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6c33c-b457-4726-8e35-17e698236d55 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:12.836615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6c33c-b457-4726-8e35-17e698236d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:12.836890 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6c33c-b457-4726-8e35-17e698236d55 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:12.837578 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f6c33c-b457-4726-8e35-17e698236d55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:12.839080 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f6c33c-b457-4726-8e35-17e698236d55 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:12.846363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 651/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jun 16 16:02:12 2022] GET /identity/v3/services => generated 2191 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:02:15.044657 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff69bb-fadb-4dea-ae14-bc20871c477e None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:15.045232 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff69bb-fadb-4dea-ae14-bc20871c477e None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:15.045232 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff69bb-fadb-4dea-ae14-bc20871c477e None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:15.046542 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 651/1302] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:02:15.052131 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b779ec-094a-4b6a-852b-b01ad8235000 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:15.052679 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b779ec-094a-4b6a-852b-b01ad8235000 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:15.052679 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b779ec-094a-4b6a-852b-b01ad8235000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:15.071149 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d6b779ec-094a-4b6a-852b-b01ad8235000 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:02:15.072361 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d6b779ec-094a-4b6a-852b-b01ad8235000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:02:15.146012 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 652/1303] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:02:15 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:02:15.170105 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff69bb-fadb-4dea-ae14-bc20871c477e None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:15.277901 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:15.285833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff69bb-fadb-4dea-ae14-bc20871c477e None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:15.323692 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3483d66d-9034-4a4a-958a-e7fd8269a4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:15.324513 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483d66d-9034-4a4a-958a-e7fd8269a4a4 None admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:15.324803 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483d66d-9034-4a4a-958a-e7fd8269a4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:15.325085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483d66d-9034-4a4a-958a-e7fd8269a4a4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:15.325759 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3483d66d-9034-4a4a-958a-e7fd8269a4a4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:15.327238 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3483d66d-9034-4a4a-958a-e7fd8269a4a4 None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:15.340787 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application [None req-3483d66d-9034-4a4a-958a-e7fd8269a4a4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 16:02:15.340787 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:15.340787 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:02:15.340787 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:02:15.340787 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:02:15.340787 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:02:15.340787 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:02:15.340787 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:02:15.340787 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 16:02:15.340787 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 16:02:15.340787 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:02:15.340787 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 16:02:15.340787 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:15.340787 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:15.340787 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:15.340787 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:15.340787 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:15.342708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:15.342708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:15.342708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:15.342708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:15.342708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:15.342708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:15.342708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:02:15.342708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:02:15.342708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:02:15.342708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:02:15.342708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:15.342708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:15.342708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:15.342708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:15.342708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:15.342708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:15.343949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:15.343949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:15.343949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:15.343949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:15.343949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:15.343949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:15.343949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:15.343949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:15.343949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:15.343949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:15.343949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:02:15.343949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:02:15.343949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 16:02:15.343949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: ERROR keystone.server.flask.application Jun 16 16:02:15.343949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 652/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jun 16 16:02:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:15.348608 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b779ec-094a-4b6a-852b-b01ad8235000 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:15.435628 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:15.459989 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b779ec-094a-4b6a-852b-b01ad8235000 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:15.508379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf6391-bbba-419d-84f9-b4cb7fd795af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:15.509510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf6391-bbba-419d-84f9-b4cb7fd795af None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:15.510021 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf6391-bbba-419d-84f9-b4cb7fd795af None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:15.510496 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf6391-bbba-419d-84f9-b4cb7fd795af None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:15.511741 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf6391-bbba-419d-84f9-b4cb7fd795af None admin] RBAC: Adding query filter params (name=placement) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:15.512167 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf6391-bbba-419d-84f9-b4cb7fd795af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:15.514228 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf6391-bbba-419d-84f9-b4cb7fd795af None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:15.523969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 653/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jun 16 16:02:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:15.529860 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3483d66d-9034-4a4a-958a-e7fd8269a4a4 None admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:15.646693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3483d66d-9034-4a4a-958a-e7fd8269a4a4 None admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:15.685152 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1285d1-b1ad-4e17-bcd0-bc028ad01cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:15.685821 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1285d1-b1ad-4e17-bcd0-bc028ad01cfc None admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:15.686134 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1285d1-b1ad-4e17-bcd0-bc028ad01cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:15.686413 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1285d1-b1ad-4e17-bcd0-bc028ad01cfc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:15.687163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1285d1-b1ad-4e17-bcd0-bc028ad01cfc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:15.688548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1285d1-b1ad-4e17-bcd0-bc028ad01cfc None admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:15.712941 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 653/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jun 16 16:02:15 2022] POST /identity/v3/endpoints => generated 331 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:02:25.518062 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=30 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:25.707489 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:22.905130 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadb3eb-990e-4275-a41d-99fd508f0b0e None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:22.905992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadb3eb-990e-4275-a41d-99fd508f0b0e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:22.905992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadb3eb-990e-4275-a41d-99fd508f0b0e None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:22.914186 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3cf175-fd74-46ff-bc1e-193b9260b4f6 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:22.915013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3cf175-fd74-46ff-bc1e-193b9260b4f6 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:22.915013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3cf175-fd74-46ff-bc1e-193b9260b4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:22.917199 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 654/1307] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:03:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:03:23.002204 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-da3cf175-fd74-46ff-bc1e-193b9260b4f6 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:03:23.003711 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-da3cf175-fd74-46ff-bc1e-193b9260b4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:23.177667 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 654/1308] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:03:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:03:23.235309 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3fb00-2bd1-4344-9184-519e437118d7 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:23.235709 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3fb00-2bd1-4344-9184-519e437118d7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:23.236093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3fb00-2bd1-4344-9184-519e437118d7 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:23.238183 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 655/1309] 10.222.0.17 () {54 vars in 964 bytes} [Thu Jun 16 16:03:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:03:23.249296 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554c1ba-41eb-41d4-94ab-bed6ca0b2d41 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:23.249914 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554c1ba-41eb-41d4-94ab-bed6ca0b2d41 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:23.250325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554c1ba-41eb-41d4-94ab-bed6ca0b2d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:23.367720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-9554c1ba-41eb-41d4-94ab-bed6ca0b2d41 None None] MFA Rules not processed for user `5a84f8aed50c4b3e8e5b33c5b587b26e`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:03:23.369498 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-9554c1ba-41eb-41d4-94ab-bed6ca0b2d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:23.394319 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:23.494526 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 655/1310] 10.222.0.17 () {58 vars in 1061 bytes} [Thu Jun 16 16:03:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:03:23.503575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3fb00-2bd1-4344-9184-519e437118d7 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:03:23.506090 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7ca3fb00-2bd1-4344-9184-519e437118d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:23.628054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3fb00-2bd1-4344-9184-519e437118d7 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:03:23.708413 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6286f24-89d2-45fd-8648-fc99bebb84d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:03:23.709178 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6286f24-89d2-45fd-8648-fc99bebb84d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:23.709526 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6286f24-89d2-45fd-8648-fc99bebb84d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:23.709870 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6286f24-89d2-45fd-8648-fc99bebb84d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:23.711644 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e6286f24-89d2-45fd-8648-fc99bebb84d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:23.768989 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:23.834872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6286f24-89d2-45fd-8648-fc99bebb84d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:03:23.836588 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6286f24-89d2-45fd-8648-fc99bebb84d8 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:03:23.856921 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 656/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:03:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:03:26.314540 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396a82a-ccd2-49c4-a46b-c9bdb83f4fc7 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:26.315089 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396a82a-ccd2-49c4-a46b-c9bdb83f4fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:26.315228 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396a82a-ccd2-49c4-a46b-c9bdb83f4fc7 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:26.316764 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 656/1312] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:03:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:03:26.322572 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644db08-73c2-439d-89a9-eb44247f6379 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:26.323090 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644db08-73c2-439d-89a9-eb44247f6379 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:26.323352 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644db08-73c2-439d-89a9-eb44247f6379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:26.414779 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-a644db08-73c2-439d-89a9-eb44247f6379 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:03:26.416094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a644db08-73c2-439d-89a9-eb44247f6379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:26.531652 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 657/1313] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:03:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:03:26.570988 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396a82a-ccd2-49c4-a46b-c9bdb83f4fc7 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:03:26.572516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-a396a82a-ccd2-49c4-a46b-c9bdb83f4fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:26.699863 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396a82a-ccd2-49c4-a46b-c9bdb83f4fc7 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:03:26.788502 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a9945-615e-4447-8021-d03ed3f6fe3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:03:26.789065 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a9945-615e-4447-8021-d03ed3f6fe3b service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:26.789651 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:26.790148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a9945-615e-4447-8021-d03ed3f6fe3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:26.790703 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a9945-615e-4447-8021-d03ed3f6fe3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:26.793356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-5a1a9945-615e-4447-8021-d03ed3f6fe3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:26.914804 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a9945-615e-4447-8021-d03ed3f6fe3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:03:26.916399 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a9945-615e-4447-8021-d03ed3f6fe3b service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:03:26.938131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 657/1314] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:03:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:03:29.467491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffd771-43ba-4fc8-bf86-1aa27a61a31a None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:29.468077 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffd771-43ba-4fc8-bf86-1aa27a61a31a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:29.468077 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffd771-43ba-4fc8-bf86-1aa27a61a31a None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:29.469526 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 658/1315] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:03:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:03:29.475830 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54f354-b5c9-44fd-be30-05e6e2f2dfc8 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:29.476276 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54f354-b5c9-44fd-be30-05e6e2f2dfc8 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:29.476419 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54f354-b5c9-44fd-be30-05e6e2f2dfc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:29.543035 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-5c54f354-b5c9-44fd-be30-05e6e2f2dfc8 None None] MFA Rules not processed for user `26f5b3d45f5048aa9a49fb86df108d90`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:03:29.544268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-5c54f354-b5c9-44fd-be30-05e6e2f2dfc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:29.596165 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:29.642046 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 658/1316] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:03:29 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:03:29.682360 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcffd771-43ba-4fc8-bf86-1aa27a61a31a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:03:29.683920 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bcffd771-43ba-4fc8-bf86-1aa27a61a31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:29.693847 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:29.791878 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcffd771-43ba-4fc8-bf86-1aa27a61a31a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:03:29.864935 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc1465-06f7-4411-b689-1b105588c081 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:03:29.865580 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc1465-06f7-4411-b689-1b105588c081 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:29.865862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc1465-06f7-4411-b689-1b105588c081 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:29.866166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc1465-06f7-4411-b689-1b105588c081 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:29.867811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-14dc1465-06f7-4411-b689-1b105588c081 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:29.932149 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:29.980766 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dc1465-06f7-4411-b689-1b105588c081 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:03:29.982401 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dc1465-06f7-4411-b689-1b105588c081 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:03:30.001541 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 659/1317] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:03:29 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:03:32.722814 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db262481-e5b8-483c-ae7d-44e7f4af6573 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:32.723377 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db262481-e5b8-483c-ae7d-44e7f4af6573 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:32.723377 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db262481-e5b8-483c-ae7d-44e7f4af6573 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:32.725307 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 659/1318] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:03:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:03:32.730734 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea3cf6-105d-4680-89c4-d31b425672eb None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:32.731162 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea3cf6-105d-4680-89c4-d31b425672eb None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:32.731298 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea3cf6-105d-4680-89c4-d31b425672eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:32.797603 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d9ea3cf6-105d-4680-89c4-d31b425672eb None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:03:32.798891 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d9ea3cf6-105d-4680-89c4-d31b425672eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:32.896973 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 660/1319] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:03:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:03:32.951063 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b04cb1-bab9-46f0-9eb3-82c3fc94ef40 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:32.951352 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b04cb1-bab9-46f0-9eb3-82c3fc94ef40 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:32.951630 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b04cb1-bab9-46f0-9eb3-82c3fc94ef40 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:32.953016 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 660/1320] 10.222.0.17 () {54 vars in 964 bytes} [Thu Jun 16 16:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:03:32.960391 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016c0aa-d0e7-480b-ae09-cc8649ad1b25 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:32.960681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016c0aa-d0e7-480b-ae09-cc8649ad1b25 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:32.960957 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016c0aa-d0e7-480b-ae09-cc8649ad1b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:32.982003 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:33.037229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-5016c0aa-d0e7-480b-ae09-cc8649ad1b25 None None] MFA Rules not processed for user `5a84f8aed50c4b3e8e5b33c5b587b26e`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:03:33.038489 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5016c0aa-d0e7-480b-ae09-cc8649ad1b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:33.118063 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 661/1321] 10.222.0.17 () {58 vars in 1061 bytes} [Thu Jun 16 16:03:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:03:33.127141 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b04cb1-bab9-46f0-9eb3-82c3fc94ef40 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:03:33.128691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-91b04cb1-bab9-46f0-9eb3-82c3fc94ef40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:33.233240 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b04cb1-bab9-46f0-9eb3-82c3fc94ef40 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:03:33.307014 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabae556-15e4-46e7-8e9a-a8ea3cf359d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:03:33.307660 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabae556-15e4-46e7-8e9a-a8ea3cf359d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:33.307946 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabae556-15e4-46e7-8e9a-a8ea3cf359d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:33.308226 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabae556-15e4-46e7-8e9a-a8ea3cf359d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:33.309939 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-dabae556-15e4-46e7-8e9a-a8ea3cf359d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:33.317337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:33.419547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabae556-15e4-46e7-8e9a-a8ea3cf359d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:03:33.421009 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabae556-15e4-46e7-8e9a-a8ea3cf359d0 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:03:33.438911 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 661/1322] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:03:33 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:03:36.771224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64141acd-523d-48bd-9ba1-5194c48281eb None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:36.771802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64141acd-523d-48bd-9ba1-5194c48281eb None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:36.771802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64141acd-523d-48bd-9ba1-5194c48281eb None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:36.773171 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 662/1323] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:03:36.778617 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996c0d3-f883-4811-9929-bdcca6011a26 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:36.779550 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996c0d3-f883-4811-9929-bdcca6011a26 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:36.780020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996c0d3-f883-4811-9929-bdcca6011a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:36.861095 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-b996c0d3-f883-4811-9929-bdcca6011a26 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:03:36.863303 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-b996c0d3-f883-4811-9929-bdcca6011a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:36.933959 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:36.982805 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 662/1324] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:03:37.018611 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64141acd-523d-48bd-9ba1-5194c48281eb None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:03:37.020852 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-64141acd-523d-48bd-9ba1-5194c48281eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:37.129451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64141acd-523d-48bd-9ba1-5194c48281eb None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:03:37.142294 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:37.209582 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054fc6ba-ca00-4c90-8982-9a30a5952326 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:03:37.210277 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fc6ba-ca00-4c90-8982-9a30a5952326 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:37.210567 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fc6ba-ca00-4c90-8982-9a30a5952326 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:37.210844 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fc6ba-ca00-4c90-8982-9a30a5952326 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:37.212550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-054fc6ba-ca00-4c90-8982-9a30a5952326 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:37.332360 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054fc6ba-ca00-4c90-8982-9a30a5952326 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:03:37.334474 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054fc6ba-ca00-4c90-8982-9a30a5952326 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:03:37.355283 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 663/1325] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:03:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:03:39.604351 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0678496-e32f-4ca0-909e-819aaa88522a None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:39.604929 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0678496-e32f-4ca0-909e-819aaa88522a None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:39.604929 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0678496-e32f-4ca0-909e-819aaa88522a None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:39.606337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 663/1326] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:03:39.612714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fa7cd-fdbd-40ab-a2e6-64d34f48b624 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:39.613322 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fa7cd-fdbd-40ab-a2e6-64d34f48b624 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:39.613322 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fa7cd-fdbd-40ab-a2e6-64d34f48b624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:39.664520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:39.683949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-fd5fa7cd-fdbd-40ab-a2e6-64d34f48b624 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:03:39.685155 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-fd5fa7cd-fdbd-40ab-a2e6-64d34f48b624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:39.794426 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 664/1327] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:03:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:03:39.830291 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0678496-e32f-4ca0-909e-819aaa88522a None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:03:39.831839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-f0678496-e32f-4ca0-909e-819aaa88522a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:39.943370 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0678496-e32f-4ca0-909e-819aaa88522a None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:03:40.020033 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a58e3-2968-47d5-a40e-31b99de88c40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:03:40.021468 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a58e3-2968-47d5-a40e-31b99de88c40 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:40.021831 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a58e3-2968-47d5-a40e-31b99de88c40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:40.022133 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a58e3-2968-47d5-a40e-31b99de88c40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:40.023852 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-158a58e3-2968-47d5-a40e-31b99de88c40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:40.041570 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:40.153007 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a58e3-2968-47d5-a40e-31b99de88c40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:03:40.155811 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a58e3-2968-47d5-a40e-31b99de88c40 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:03:40.175699 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 664/1328] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:03:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:03:42.430920 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b095a6-7344-4fad-ac9f-b0b804cd9e2d None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:42.431463 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b095a6-7344-4fad-ac9f-b0b804cd9e2d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:42.431463 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b095a6-7344-4fad-ac9f-b0b804cd9e2d None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:42.432862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 665/1329] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:03:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:03:42.438825 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359f085-6494-48b7-9f81-b3b961095350 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:42.439591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359f085-6494-48b7-9f81-b3b961095350 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:42.439881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359f085-6494-48b7-9f81-b3b961095350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:42.521232 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-0359f085-6494-48b7-9f81-b3b961095350 None None] MFA Rules not processed for user `26f5b3d45f5048aa9a49fb86df108d90`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:03:42.522415 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-0359f085-6494-48b7-9f81-b3b961095350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:42.603922 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:42.643929 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 665/1330] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:03:42 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:03:42.683783 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b095a6-7344-4fad-ac9f-b0b804cd9e2d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:03:42.685408 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-18b095a6-7344-4fad-ac9f-b0b804cd9e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:42.837642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b095a6-7344-4fad-ac9f-b0b804cd9e2d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:03:42.858946 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:42.920837 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb19483-e61e-472c-8d6d-33a15def29bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:03:42.921528 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb19483-e61e-472c-8d6d-33a15def29bd service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:42.921803 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb19483-e61e-472c-8d6d-33a15def29bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:42.922119 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb19483-e61e-472c-8d6d-33a15def29bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:42.924178 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8cb19483-e61e-472c-8d6d-33a15def29bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:43.048489 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb19483-e61e-472c-8d6d-33a15def29bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:03:43.051073 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb19483-e61e-472c-8d6d-33a15def29bd service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:03:43.073506 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 666/1331] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:03:42 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:03:45.508226 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05be223-7b1a-4165-ad76-70304c3c4456 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:45.509137 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05be223-7b1a-4165-ad76-70304c3c4456 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:45.509137 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05be223-7b1a-4165-ad76-70304c3c4456 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:45.510448 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 666/1332] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:03:45.515616 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f2fd6-038f-4da8-bb8d-ccb735246d2c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:45.516049 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f2fd6-038f-4da8-bb8d-ccb735246d2c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:45.516171 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f2fd6-038f-4da8-bb8d-ccb735246d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:45.579383 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:45.583963 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-784f2fd6-038f-4da8-bb8d-ccb735246d2c None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:03:45.585554 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-784f2fd6-038f-4da8-bb8d-ccb735246d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:45.684741 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 667/1333] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:03:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:03:45.720365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05be223-7b1a-4165-ad76-70304c3c4456 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:03:45.721976 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-f05be223-7b1a-4165-ad76-70304c3c4456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:45.831732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05be223-7b1a-4165-ad76-70304c3c4456 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:03:45.908425 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fda9d4-413d-49e7-bdd6-ebf93d99472e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:03:45.909075 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fda9d4-413d-49e7-bdd6-ebf93d99472e service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:45.909388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fda9d4-413d-49e7-bdd6-ebf93d99472e service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:45.909665 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fda9d4-413d-49e7-bdd6-ebf93d99472e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:45.911664 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-15fda9d4-413d-49e7-bdd6-ebf93d99472e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:45.940017 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:46.023964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fda9d4-413d-49e7-bdd6-ebf93d99472e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:03:46.025426 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fda9d4-413d-49e7-bdd6-ebf93d99472e service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:03:46.043393 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 667/1334] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:03:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:03:48.272523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32199561-3796-4f6a-8e6f-384ae5afa8a4 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:48.273116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32199561-3796-4f6a-8e6f-384ae5afa8a4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:48.273116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32199561-3796-4f6a-8e6f-384ae5afa8a4 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:48.274520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 668/1335] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:03:48.279812 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fba46-aa80-44f1-a2bc-a8501935068a None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:48.280260 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fba46-aa80-44f1-a2bc-a8501935068a None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:48.280362 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fba46-aa80-44f1-a2bc-a8501935068a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:48.344954 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-c71fba46-aa80-44f1-a2bc-a8501935068a None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:03:48.346277 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-c71fba46-aa80-44f1-a2bc-a8501935068a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:48.421172 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:48.444244 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 668/1336] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:03:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:03:48.486230 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32199561-3796-4f6a-8e6f-384ae5afa8a4 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:03:48.487804 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-32199561-3796-4f6a-8e6f-384ae5afa8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:48.595430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32199561-3796-4f6a-8e6f-384ae5afa8a4 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:03:48.625540 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:48.673076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef276dbf-28b3-4ae9-9169-58bb8d55c26d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:03:48.673729 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef276dbf-28b3-4ae9-9169-58bb8d55c26d service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:48.674045 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef276dbf-28b3-4ae9-9169-58bb8d55c26d service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:48.674319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef276dbf-28b3-4ae9-9169-58bb8d55c26d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:48.675971 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ef276dbf-28b3-4ae9-9169-58bb8d55c26d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:48.786353 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef276dbf-28b3-4ae9-9169-58bb8d55c26d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:03:48.787865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef276dbf-28b3-4ae9-9169-58bb8d55c26d service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:03:48.806997 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 669/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:03:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:03:51.418079 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b003e5-bfa7-4ed9-b9c6-c6d17042ba8c None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:51.418668 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b003e5-bfa7-4ed9-b9c6-c6d17042ba8c None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:51.418668 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b003e5-bfa7-4ed9-b9c6-c6d17042ba8c None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:51.420023 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 669/1338] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:03:51.427375 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ac785-cd78-4ba5-adc5-0567f3824a56 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:51.427867 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ac785-cd78-4ba5-adc5-0567f3824a56 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:51.428112 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ac785-cd78-4ba5-adc5-0567f3824a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:51.495689 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8c7ac785-cd78-4ba5-adc5-0567f3824a56 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:03:51.496985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8c7ac785-cd78-4ba5-adc5-0567f3824a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:51.506249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:51.596636 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 670/1339] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:03:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:03:51.634145 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b003e5-bfa7-4ed9-b9c6-c6d17042ba8c None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:03:51.635763 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-a6b003e5-bfa7-4ed9-b9c6-c6d17042ba8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:51.741060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b003e5-bfa7-4ed9-b9c6-c6d17042ba8c None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:03:51.815293 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eef695-a22e-4434-9034-092ab7fc09ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:03:51.815951 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eef695-a22e-4434-9034-092ab7fc09ac service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:51.816237 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eef695-a22e-4434-9034-092ab7fc09ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:51.816516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eef695-a22e-4434-9034-092ab7fc09ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:51.818370 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-97eef695-a22e-4434-9034-092ab7fc09ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:51.855096 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:51.929370 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eef695-a22e-4434-9034-092ab7fc09ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:03:51.930846 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eef695-a22e-4434-9034-092ab7fc09ac service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:03:51.948903 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 670/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:03:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:03:56.051264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dfaf4-7aa1-4e74-a694-647be6edb0ea None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:56.051893 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dfaf4-7aa1-4e74-a694-647be6edb0ea None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:56.051893 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dfaf4-7aa1-4e74-a694-647be6edb0ea None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:56.053445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 671/1341] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:03:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:03:56.058987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d2987-0ac5-4ab7-8d79-a4bfa00d829c None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:56.059461 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d2987-0ac5-4ab7-8d79-a4bfa00d829c None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:56.059675 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d2987-0ac5-4ab7-8d79-a4bfa00d829c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:56.133736 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-1d8d2987-0ac5-4ab7-8d79-a4bfa00d829c None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:03:56.134999 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-1d8d2987-0ac5-4ab7-8d79-a4bfa00d829c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:56.231200 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:56.245816 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 671/1342] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:03:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:03:56.285401 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165dfaf4-7aa1-4e74-a694-647be6edb0ea None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:03:56.288260 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-165dfaf4-7aa1-4e74-a694-647be6edb0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:56.388147 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165dfaf4-7aa1-4e74-a694-647be6edb0ea None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:03:56.391660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba4ec8-38fd-48a7-bedb-7f83fe7b66e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:03:56.392314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba4ec8-38fd-48a7-bedb-7f83fe7b66e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:56.392602 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba4ec8-38fd-48a7-bedb-7f83fe7b66e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:56.392889 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba4ec8-38fd-48a7-bedb-7f83fe7b66e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:56.402678 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba4ec8-38fd-48a7-bedb-7f83fe7b66e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:03:56.404229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba4ec8-38fd-48a7-bedb-7f83fe7b66e9 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:03:56.415049 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 672/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:03:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:03:59.227111 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80d918-1a33-436c-a2e1-91021754781a None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:59.227655 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80d918-1a33-436c-a2e1-91021754781a None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:59.227788 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80d918-1a33-436c-a2e1-91021754781a None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:59.229216 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 672/1344] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:03:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:03:59.234729 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934ecc7-5856-4f9d-b764-be7b0a9173a6 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:59.235211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934ecc7-5856-4f9d-b764-be7b0a9173a6 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:59.235351 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934ecc7-5856-4f9d-b764-be7b0a9173a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:59.266633 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-6934ecc7-5856-4f9d-b764-be7b0a9173a6 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:03:59.267865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6934ecc7-5856-4f9d-b764-be7b0a9173a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:59.296787 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:59.353903 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 673/1345] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:03:59 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:03:59.391976 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80d918-1a33-436c-a2e1-91021754781a None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:03:59.406467 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80d918-1a33-436c-a2e1-91021754781a None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:03:59.411581 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aebf8a-1eaa-45eb-8e96-b0ae73e2a42a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:03:59.412241 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aebf8a-1eaa-45eb-8e96-b0ae73e2a42a service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:59.412528 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aebf8a-1eaa-45eb-8e96-b0ae73e2a42a service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:59.412884 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aebf8a-1eaa-45eb-8e96-b0ae73e2a42a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:59.424585 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aebf8a-1eaa-45eb-8e96-b0ae73e2a42a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:03:59.428302 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aebf8a-1eaa-45eb-8e96-b0ae73e2a42a service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:03:59.443846 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 673/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:03:59 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:04:03.789949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc180918-d961-49f5-a4b8-38f022ccba6b None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:03.790371 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc180918-d961-49f5-a4b8-38f022ccba6b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:03.790543 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc180918-d961-49f5-a4b8-38f022ccba6b None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:03.791909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 674/1347] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:03.797501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ecf4f8-a8f5-451e-b37d-af0e1c4a3d36 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:03.797501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ecf4f8-a8f5-451e-b37d-af0e1c4a3d36 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:03.798146 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ecf4f8-a8f5-451e-b37d-af0e1c4a3d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:03.817790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-f2ecf4f8-a8f5-451e-b37d-af0e1c4a3d36 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:03.819042 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-f2ecf4f8-a8f5-451e-b37d-af0e1c4a3d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:03.827161 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 674/1348] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:04:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:03.866755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc180918-d961-49f5-a4b8-38f022ccba6b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:03.868514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bc180918-d961-49f5-a4b8-38f022ccba6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:03.882826 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc180918-d961-49f5-a4b8-38f022ccba6b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:03.886562 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffa9bd-8803-4873-a0a4-8cfc0989ceb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:03.887200 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffa9bd-8803-4873-a0a4-8cfc0989ceb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:03.887487 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffa9bd-8803-4873-a0a4-8cfc0989ceb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:03.887757 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffa9bd-8803-4873-a0a4-8cfc0989ceb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:03.897828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dffa9bd-8803-4873-a0a4-8cfc0989ceb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:03.899355 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dffa9bd-8803-4873-a0a4-8cfc0989ceb0 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:03.909316 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 675/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:04:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:04:09.097770 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c28482-1924-4ae4-b4e7-ced43d388038 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:09.099317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c28482-1924-4ae4-b4e7-ced43d388038 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:09.100046 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c28482-1924-4ae4-b4e7-ced43d388038 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:09.101736 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 675/1350] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jun 16 16:04:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:09.112808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820debf-f014-4d31-bdae-518315fb35cc None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:09.113426 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820debf-f014-4d31-bdae-518315fb35cc None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:09.113426 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820debf-f014-4d31-bdae-518315fb35cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:09.196335 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b820debf-f014-4d31-bdae-518315fb35cc None None] MFA Rules not processed for user `36b63614297346e29ff276eab444fe5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:09.197538 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b820debf-f014-4d31-bdae-518315fb35cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:09.276492 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 676/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jun 16 16:04:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:13.651167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7cc32-7325-4c6f-92b0-57b5577af9e6 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:13.652148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7cc32-7325-4c6f-92b0-57b5577af9e6 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:13.652631 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7cc32-7325-4c6f-92b0-57b5577af9e6 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:13.654562 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 676/1352] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jun 16 16:04:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:13.659711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d29952-049d-402e-9de6-0959782c3403 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:13.660258 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d29952-049d-402e-9de6-0959782c3403 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:13.660505 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d29952-049d-402e-9de6-0959782c3403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:13.679707 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-72d29952-049d-402e-9de6-0959782c3403 None None] MFA Rules not processed for user `36b63614297346e29ff276eab444fe5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:13.680912 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-72d29952-049d-402e-9de6-0959782c3403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:13.690594 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 677/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jun 16 16:04:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:13.812128 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=13 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:15.190402 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fab34d-c4e8-45eb-9f21-73edad8f8777 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:15.191010 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fab34d-c4e8-45eb-9f21-73edad8f8777 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:15.191010 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fab34d-c4e8-45eb-9f21-73edad8f8777 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:15.192916 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 677/1354] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jun 16 16:04:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:15.198179 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d0288-fc20-4638-a96a-3540bd266213 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:15.198659 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d0288-fc20-4638-a96a-3540bd266213 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:15.198770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d0288-fc20-4638-a96a-3540bd266213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:15.221613 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d66d0288-fc20-4638-a96a-3540bd266213 None None] MFA Rules not processed for user `36b63614297346e29ff276eab444fe5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:15.222955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d66d0288-fc20-4638-a96a-3540bd266213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:15.231862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 678/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jun 16 16:04:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:19.334033 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013c7be-d8e5-4104-8fd9-006a59b84d49 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:19.334650 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013c7be-d8e5-4104-8fd9-006a59b84d49 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:19.334650 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013c7be-d8e5-4104-8fd9-006a59b84d49 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:19.340304 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 678/1356] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:04:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:19.341399 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cee581-9651-41e1-b2c5-98ff1e40ff6e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:19.342303 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cee581-9651-41e1-b2c5-98ff1e40ff6e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:19.342807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cee581-9651-41e1-b2c5-98ff1e40ff6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:19.361711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b7cee581-9651-41e1-b2c5-98ff1e40ff6e None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:19.363299 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b7cee581-9651-41e1-b2c5-98ff1e40ff6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:19.371483 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 679/1357] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:04:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:04:19.426818 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2751ad35-8d25-4e3d-b79c-f1c39cacc5a8 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:19.427525 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2751ad35-8d25-4e3d-b79c-f1c39cacc5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:19.427983 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2751ad35-8d25-4e3d-b79c-f1c39cacc5a8 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:19.430132 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 679/1358] 10.222.0.17 () {54 vars in 961 bytes} [Thu Jun 16 16:04:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:19.439387 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a99162-15a3-486e-a1c6-dff5bb78e8bd None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:19.440056 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a99162-15a3-486e-a1c6-dff5bb78e8bd None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:19.440509 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a99162-15a3-486e-a1c6-dff5bb78e8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:19.493361 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:19.502652 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-01a99162-15a3-486e-a1c6-dff5bb78e8bd None None] MFA Rules not processed for user `c3645b1a0be34c01ac87c734fcd85ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:19.504391 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-01a99162-15a3-486e-a1c6-dff5bb78e8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:19.598654 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 680/1359] 10.222.0.17 () {58 vars in 1058 bytes} [Thu Jun 16 16:04:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:19.607087 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2751ad35-8d25-4e3d-b79c-f1c39cacc5a8 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:19.626155 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2751ad35-8d25-4e3d-b79c-f1c39cacc5a8 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:19.635319 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800586b-b527-4e96-ab5c-4328bf47a820 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:19.636081 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800586b-b527-4e96-ab5c-4328bf47a820 service nova] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:19.636462 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800586b-b527-4e96-ab5c-4328bf47a820 service nova] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:19.636834 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800586b-b527-4e96-ab5c-4328bf47a820 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:19.646494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800586b-b527-4e96-ab5c-4328bf47a820 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:19.648242 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800586b-b527-4e96-ab5c-4328bf47a820 service nova] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:19.659087 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 680/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:04:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:04:22.082495 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a7ea9-705f-47b6-9594-433b6b468ece None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:22.083101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a7ea9-705f-47b6-9594-433b6b468ece None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:22.083414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a7ea9-705f-47b6-9594-433b6b468ece None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:22.085295 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 681/1361] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:04:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:22.091367 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c003764-1458-4922-92da-2b61a214106a None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:22.091954 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c003764-1458-4922-92da-2b61a214106a None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:22.092166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c003764-1458-4922-92da-2b61a214106a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:22.113986 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-3c003764-1458-4922-92da-2b61a214106a None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:22.115212 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-3c003764-1458-4922-92da-2b61a214106a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:22.126738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 681/1362] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:22.173810 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a7ea9-705f-47b6-9594-433b6b468ece None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:22.190368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a7ea9-705f-47b6-9594-433b6b468ece None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:22.196948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a33f6b-f770-4213-8ab1-26d34cd92c20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:22.197634 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a33f6b-f770-4213-8ab1-26d34cd92c20 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:22.197941 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a33f6b-f770-4213-8ab1-26d34cd92c20 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:22.198224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a33f6b-f770-4213-8ab1-26d34cd92c20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:22.208209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a33f6b-f770-4213-8ab1-26d34cd92c20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:22.210047 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a33f6b-f770-4213-8ab1-26d34cd92c20 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:22.220871 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 682/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:04:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:04:24.804225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855369d-178f-49ec-a795-cd3f9d0c7ae9 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:24.804717 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855369d-178f-49ec-a795-cd3f9d0c7ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:24.804810 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855369d-178f-49ec-a795-cd3f9d0c7ae9 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:24.806189 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 682/1364] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:24.811353 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e6c30-3466-4cbc-a6e8-5cfea3072c43 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:24.811786 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e6c30-3466-4cbc-a6e8-5cfea3072c43 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:24.811901 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e6c30-3466-4cbc-a6e8-5cfea3072c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:24.830911 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-473e6c30-3466-4cbc-a6e8-5cfea3072c43 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:24.832376 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-473e6c30-3466-4cbc-a6e8-5cfea3072c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:24.846312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 683/1365] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:24.895070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84250155-cc39-4df8-98fb-d2bb69574e34 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:24.895362 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84250155-cc39-4df8-98fb-d2bb69574e34 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:24.895645 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84250155-cc39-4df8-98fb-d2bb69574e34 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:24.897023 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 683/1366] 10.222.0.17 () {54 vars in 961 bytes} [Thu Jun 16 16:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:24.903027 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72f0c9-4466-4ef6-8188-0e57966f6a70 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:24.903323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72f0c9-4466-4ef6-8188-0e57966f6a70 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:24.903604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72f0c9-4466-4ef6-8188-0e57966f6a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:24.922827 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9e72f0c9-4466-4ef6-8188-0e57966f6a70 None None] MFA Rules not processed for user `c3645b1a0be34c01ac87c734fcd85ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:24.924031 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9e72f0c9-4466-4ef6-8188-0e57966f6a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:24.953559 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 684/1367] 10.222.0.17 () {58 vars in 1058 bytes} [Thu Jun 16 16:04:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:24.960121 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84250155-cc39-4df8-98fb-d2bb69574e34 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:24.981702 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84250155-cc39-4df8-98fb-d2bb69574e34 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:24.988549 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465abc24-3591-4dd4-94db-16f0d4e129af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:24.989493 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465abc24-3591-4dd4-94db-16f0d4e129af service nova] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:24.990091 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465abc24-3591-4dd4-94db-16f0d4e129af service nova] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:24.990597 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465abc24-3591-4dd4-94db-16f0d4e129af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:25.000153 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465abc24-3591-4dd4-94db-16f0d4e129af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:25.001777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465abc24-3591-4dd4-94db-16f0d4e129af service nova] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:25.013495 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 684/1368] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:04:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:04:27.252962 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3bed31-e115-4707-aeca-735c9cb52342 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:27.254010 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3bed31-e115-4707-aeca-735c9cb52342 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:27.254129 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3bed31-e115-4707-aeca-735c9cb52342 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:27.255861 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 685/1369] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:04:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:27.261493 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2f0ba-c585-479e-96dc-1413148cb31e None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:27.262306 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2f0ba-c585-479e-96dc-1413148cb31e None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:27.262738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2f0ba-c585-479e-96dc-1413148cb31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:27.282149 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-60c2f0ba-c585-479e-96dc-1413148cb31e None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:27.283578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-60c2f0ba-c585-479e-96dc-1413148cb31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:27.292420 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 685/1370] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:27.339361 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3bed31-e115-4707-aeca-735c9cb52342 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:27.355403 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3bed31-e115-4707-aeca-735c9cb52342 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:27.361689 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86418972-2d63-410d-ab88-564f38f0b4e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:27.362436 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86418972-2d63-410d-ab88-564f38f0b4e9 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:27.362845 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86418972-2d63-410d-ab88-564f38f0b4e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:27.363797 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86418972-2d63-410d-ab88-564f38f0b4e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:27.373764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86418972-2d63-410d-ab88-564f38f0b4e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:27.375691 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86418972-2d63-410d-ab88-564f38f0b4e9 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:27.389161 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 686/1371] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:04:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:04:27.714275 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b5833-b831-416d-a09c-5819169e86a2 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:27.714568 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b5833-b831-416d-a09c-5819169e86a2 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:27.714950 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b5833-b831-416d-a09c-5819169e86a2 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:27.716408 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 686/1372] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:27.722118 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffdc0e-5609-47a8-ab42-13e0c5a66a3c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:27.723165 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffdc0e-5609-47a8-ab42-13e0c5a66a3c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:27.723522 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffdc0e-5609-47a8-ab42-13e0c5a66a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:27.742632 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e5ffdc0e-5609-47a8-ab42-13e0c5a66a3c None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:27.743829 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e5ffdc0e-5609-47a8-ab42-13e0c5a66a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:27.751889 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 687/1373] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:28.031050 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a3405-2b73-463a-9ca7-ef5b726b9930 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:28.031331 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a3405-2b73-463a-9ca7-ef5b726b9930 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:28.031797 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a3405-2b73-463a-9ca7-ef5b726b9930 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:28.033168 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 687/1374] 10.222.0.17 () {54 vars in 963 bytes} [Thu Jun 16 16:04:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:28.039460 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d45e7-2715-46d3-803b-77e31bf293cd None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:28.039831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d45e7-2715-46d3-803b-77e31bf293cd None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:28.040174 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d45e7-2715-46d3-803b-77e31bf293cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:28.093128 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9b4d45e7-2715-46d3-803b-77e31bf293cd None None] MFA Rules not processed for user `69b669a5ba4e4294baeea369949e3e6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:28.094374 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9b4d45e7-2715-46d3-803b-77e31bf293cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:28.160662 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 688/1375] 10.222.0.17 () {58 vars in 1060 bytes} [Thu Jun 16 16:04:28 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:28.168707 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a3405-2b73-463a-9ca7-ef5b726b9930 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:28.181401 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a3405-2b73-463a-9ca7-ef5b726b9930 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:28.185193 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e88d5-bd94-483f-8716-db088b361fd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:28.185846 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e88d5-bd94-483f-8716-db088b361fd7 service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:28.186168 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e88d5-bd94-483f-8716-db088b361fd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:28.186445 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e88d5-bd94-483f-8716-db088b361fd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:28.196802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2e88d5-bd94-483f-8716-db088b361fd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:28.198967 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2e88d5-bd94-483f-8716-db088b361fd7 service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:28.210300 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 688/1376] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:04:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:04:29.646052 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c69cf2-7929-476e-8716-88314545b6af None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:29.646648 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c69cf2-7929-476e-8716-88314545b6af None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:29.646648 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c69cf2-7929-476e-8716-88314545b6af None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:29.647983 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 689/1377] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:04:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:29.653315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3a7d9-e9e8-43fc-a127-e37c8d172283 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:29.653715 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3a7d9-e9e8-43fc-a127-e37c8d172283 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:29.653856 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3a7d9-e9e8-43fc-a127-e37c8d172283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:29.673278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-3ad3a7d9-e9e8-43fc-a127-e37c8d172283 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:29.674631 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-3ad3a7d9-e9e8-43fc-a127-e37c8d172283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:29.683053 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 689/1378] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:29.727858 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c69cf2-7929-476e-8716-88314545b6af None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:29.743985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c69cf2-7929-476e-8716-88314545b6af None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:29.750214 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214a3ce-8612-4de2-827d-400509cefe4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:29.750925 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214a3ce-8612-4de2-827d-400509cefe4d service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:29.751219 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214a3ce-8612-4de2-827d-400509cefe4d service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:29.751510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214a3ce-8612-4de2-827d-400509cefe4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:29.762387 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b214a3ce-8612-4de2-827d-400509cefe4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:29.764153 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b214a3ce-8612-4de2-827d-400509cefe4d service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:29.769070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:29.777648 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 690/1379] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:04:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:04:32.029997 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af456d-2e41-420d-8ade-782a114af6e5 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:32.030522 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af456d-2e41-420d-8ade-782a114af6e5 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:32.030522 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af456d-2e41-420d-8ade-782a114af6e5 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:32.032045 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 690/1380] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:32.037540 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeaf7c0-c24f-4809-bace-dd8585d605d8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:32.038093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeaf7c0-c24f-4809-bace-dd8585d605d8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:32.038177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeaf7c0-c24f-4809-bace-dd8585d605d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:32.059596 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ffeaf7c0-c24f-4809-bace-dd8585d605d8 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:32.061111 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ffeaf7c0-c24f-4809-bace-dd8585d605d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:32.070158 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 691/1381] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:04:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:32.115982 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af456d-2e41-420d-8ade-782a114af6e5 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:32.131745 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af456d-2e41-420d-8ade-782a114af6e5 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:32.138085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72060e00-695f-40ea-91b8-ecab84837ece service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:32.138847 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72060e00-695f-40ea-91b8-ecab84837ece service nova] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:32.139236 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72060e00-695f-40ea-91b8-ecab84837ece service nova] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:32.139618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72060e00-695f-40ea-91b8-ecab84837ece service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:32.150872 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72060e00-695f-40ea-91b8-ecab84837ece service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:32.152585 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72060e00-695f-40ea-91b8-ecab84837ece service nova] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:32.163457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 691/1382] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:04:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:04:34.408005 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6daada-6190-489d-9d16-a5b0d3ef73c9 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:34.408631 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6daada-6190-489d-9d16-a5b0d3ef73c9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:34.408631 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6daada-6190-489d-9d16-a5b0d3ef73c9 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:34.410786 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 692/1383] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:04:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:04:34.416681 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb6aab-6d06-478f-a61d-b34a4d9d60ad None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:34.417401 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb6aab-6d06-478f-a61d-b34a4d9d60ad None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:34.417401 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb6aab-6d06-478f-a61d-b34a4d9d60ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:34.436963 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-efeb6aab-6d06-478f-a61d-b34a4d9d60ad None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:34.437799 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-efeb6aab-6d06-478f-a61d-b34a4d9d60ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:34.447488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 692/1384] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:04:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:34.495924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6daada-6190-489d-9d16-a5b0d3ef73c9 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:34.512325 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6daada-6190-489d-9d16-a5b0d3ef73c9 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:34.518376 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbed55d-b8ce-4eb4-b17c-152c4d66d59c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:34.519246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbed55d-b8ce-4eb4-b17c-152c4d66d59c service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:34.519530 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbed55d-b8ce-4eb4-b17c-152c4d66d59c service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:34.520605 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbed55d-b8ce-4eb4-b17c-152c4d66d59c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:34.529377 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbed55d-b8ce-4eb4-b17c-152c4d66d59c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:34.531162 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbed55d-b8ce-4eb4-b17c-152c4d66d59c service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:34.541845 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 693/1385] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:04:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:04:36.748832 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb67b2-a0c3-4643-bbc1-d7bb23ec457a None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:36.749440 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb67b2-a0c3-4643-bbc1-d7bb23ec457a None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:36.749440 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb67b2-a0c3-4643-bbc1-d7bb23ec457a None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:36.750911 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 693/1386] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:36.756751 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e948fc6-c714-4c81-bae1-aa7d79fbb0e3 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:36.757335 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e948fc6-c714-4c81-bae1-aa7d79fbb0e3 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:36.757335 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e948fc6-c714-4c81-bae1-aa7d79fbb0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:36.776299 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8e948fc6-c714-4c81-bae1-aa7d79fbb0e3 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:36.777527 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8e948fc6-c714-4c81-bae1-aa7d79fbb0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:36.789412 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 694/1387] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:04:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:36.834433 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb67b2-a0c3-4643-bbc1-d7bb23ec457a None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:36.849384 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb67b2-a0c3-4643-bbc1-d7bb23ec457a None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:36.855126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd26ab-31a2-46eb-96d5-afc45d0de6f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:36.855843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd26ab-31a2-46eb-96d5-afc45d0de6f7 service nova] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:36.856070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd26ab-31a2-46eb-96d5-afc45d0de6f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:36.856350 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd26ab-31a2-46eb-96d5-afc45d0de6f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:36.865786 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd26ab-31a2-46eb-96d5-afc45d0de6f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:36.867432 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd26ab-31a2-46eb-96d5-afc45d0de6f7 service nova] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:36.877230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 694/1388] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:04:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:04:39.141591 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff8a1c-77d9-41a3-9c98-48cd7f673ed5 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:39.142461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff8a1c-77d9-41a3-9c98-48cd7f673ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:39.142461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff8a1c-77d9-41a3-9c98-48cd7f673ed5 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:39.143653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 695/1389] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:04:39.150301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9855a26-9ddd-45c6-b13e-935034100a0f None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:39.151402 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9855a26-9ddd-45c6-b13e-935034100a0f None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:39.151402 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9855a26-9ddd-45c6-b13e-935034100a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:39.172059 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-b9855a26-9ddd-45c6-b13e-935034100a0f None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:39.173320 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-b9855a26-9ddd-45c6-b13e-935034100a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:39.182394 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 695/1390] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:39.233052 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff8a1c-77d9-41a3-9c98-48cd7f673ed5 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:39.249674 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff8a1c-77d9-41a3-9c98-48cd7f673ed5 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:39.256258 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6df54b-9def-4d7c-a2da-f2275586bd4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:39.257362 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6df54b-9def-4d7c-a2da-f2275586bd4d service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:39.257725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6df54b-9def-4d7c-a2da-f2275586bd4d service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:39.258192 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6df54b-9def-4d7c-a2da-f2275586bd4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:39.273787 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6df54b-9def-4d7c-a2da-f2275586bd4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:39.276553 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6df54b-9def-4d7c-a2da-f2275586bd4d service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:39.291362 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 696/1391] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:04:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:04:41.746368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f9010-fedb-4360-825d-f44e6d418a8f None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:41.747251 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f9010-fedb-4360-825d-f44e6d418a8f None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:41.748317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f9010-fedb-4360-825d-f44e6d418a8f None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:41.749773 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 696/1392] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:04:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:41.756185 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f71dbe-a910-4321-b905-53b219e9e18d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:41.756688 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f71dbe-a910-4321-b905-53b219e9e18d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:41.756803 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f71dbe-a910-4321-b905-53b219e9e18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:41.777634 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-19f71dbe-a910-4321-b905-53b219e9e18d None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:41.778829 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-19f71dbe-a910-4321-b905-53b219e9e18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:41.789702 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 697/1393] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:04:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:41.838041 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247f9010-fedb-4360-825d-f44e6d418a8f None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:41.855018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247f9010-fedb-4360-825d-f44e6d418a8f None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:41.860561 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35476b81-dd95-4a42-84dd-5421f537f2d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:41.861216 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35476b81-dd95-4a42-84dd-5421f537f2d8 service nova] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:41.861494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35476b81-dd95-4a42-84dd-5421f537f2d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:41.861773 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35476b81-dd95-4a42-84dd-5421f537f2d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:41.872250 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35476b81-dd95-4a42-84dd-5421f537f2d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:41.873886 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35476b81-dd95-4a42-84dd-5421f537f2d8 service nova] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:41.885269 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 697/1394] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:04:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:04:43.843676 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e41880-d012-4557-b217-eca1ab400192 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:43.844237 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e41880-d012-4557-b217-eca1ab400192 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:43.844237 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e41880-d012-4557-b217-eca1ab400192 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:43.845611 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 698/1395] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:04:43.851332 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d558fe-64fd-47f1-aa40-f66bae86518a None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:43.851750 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d558fe-64fd-47f1-aa40-f66bae86518a None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:43.851932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d558fe-64fd-47f1-aa40-f66bae86518a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:43.858352 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=52 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:43.872089 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-73d558fe-64fd-47f1-aa40-f66bae86518a None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:43.873301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-73d558fe-64fd-47f1-aa40-f66bae86518a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:43.881251 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 698/1396] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:04:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:43.938054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53ef08-74cd-4d36-a2e0-fd786e87f78a None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:43.938341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53ef08-74cd-4d36-a2e0-fd786e87f78a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:43.938624 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53ef08-74cd-4d36-a2e0-fd786e87f78a None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:43.939992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 699/1397] 10.222.0.17 () {54 vars in 962 bytes} [Thu Jun 16 16:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:43.946166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d15207-6fd8-42c5-801a-c8547e65e611 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:43.946528 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d15207-6fd8-42c5-801a-c8547e65e611 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:43.946874 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d15207-6fd8-42c5-801a-c8547e65e611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:43.998310 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-b1d15207-6fd8-42c5-801a-c8547e65e611 None None] MFA Rules not processed for user `6f11beca9c5148eea4cc45f8dc89dbda`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:43.999565 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-b1d15207-6fd8-42c5-801a-c8547e65e611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:44.066031 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 699/1398] 10.222.0.17 () {58 vars in 1059 bytes} [Thu Jun 16 16:04:43 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:44.074179 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee53ef08-74cd-4d36-a2e0-fd786e87f78a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:44.087039 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee53ef08-74cd-4d36-a2e0-fd786e87f78a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:44.091882 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19343e9f-32c7-408c-86bb-37a7e43d47b2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f11beca9c5148eea4cc45f8dc89dbda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:44.092593 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19343e9f-32c7-408c-86bb-37a7e43d47b2 service glance] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:44.092879 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19343e9f-32c7-408c-86bb-37a7e43d47b2 service glance] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:44.093162 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19343e9f-32c7-408c-86bb-37a7e43d47b2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:44.103167 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19343e9f-32c7-408c-86bb-37a7e43d47b2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:44.104613 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19343e9f-32c7-408c-86bb-37a7e43d47b2 service glance] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:44.115000 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 700/1399] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:04:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:04:44.138149 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096c633-5d57-4be7-92c8-8cf21fd67071 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:44.138448 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096c633-5d57-4be7-92c8-8cf21fd67071 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:44.138734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096c633-5d57-4be7-92c8-8cf21fd67071 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:44.140212 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 700/1400] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:44.146195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba71e5-ae9c-45e1-a6b5-20f26da1bc5b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:44.146492 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba71e5-ae9c-45e1-a6b5-20f26da1bc5b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:44.146774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba71e5-ae9c-45e1-a6b5-20f26da1bc5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:44.166156 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-eeba71e5-ae9c-45e1-a6b5-20f26da1bc5b None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:44.167711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-eeba71e5-ae9c-45e1-a6b5-20f26da1bc5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:44.176614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 701/1401] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:04:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:04:44.225413 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096c633-5d57-4be7-92c8-8cf21fd67071 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:44.241755 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096c633-5d57-4be7-92c8-8cf21fd67071 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:44.252094 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc6b276-3cc2-4151-82c1-6009f4686057 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:44.252745 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6b276-3cc2-4151-82c1-6009f4686057 service nova] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:44.253864 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6b276-3cc2-4151-82c1-6009f4686057 service nova] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:44.254306 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6b276-3cc2-4151-82c1-6009f4686057 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:44.264036 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc6b276-3cc2-4151-82c1-6009f4686057 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:44.266338 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc6b276-3cc2-4151-82c1-6009f4686057 service nova] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:44.280621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 701/1402] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:04:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:04:44.871854 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373fd57-39ae-47e6-91c4-2fd53d3fc374 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:44.872447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373fd57-39ae-47e6-91c4-2fd53d3fc374 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:44.872447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373fd57-39ae-47e6-91c4-2fd53d3fc374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:44.924456 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e373fd57-39ae-47e6-91c4-2fd53d3fc374 None None] MFA Rules not processed for user `2c2e0e19880f4b048faf034dd38c7365`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:44.926204 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e373fd57-39ae-47e6-91c4-2fd53d3fc374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:44.998756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 702/1403] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 16:04:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:45.042199 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186e4bd-3d62-47bf-b187-2006a4477e91 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:45.042753 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186e4bd-3d62-47bf-b187-2006a4477e91 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:45.042879 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186e4bd-3d62-47bf-b187-2006a4477e91 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:45.044227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 702/1404] 10.222.0.17 () {54 vars in 942 bytes} [Thu Jun 16 16:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:45.051863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cedd0d-e2fc-4761-bc47-5d6d65273f3b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:45.052227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cedd0d-e2fc-4761-bc47-5d6d65273f3b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:45.052579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cedd0d-e2fc-4761-bc47-5d6d65273f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:45.104010 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-57cedd0d-e2fc-4761-bc47-5d6d65273f3b None None] MFA Rules not processed for user `0082e37ce54f49f7837601349508c89f`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:45.105335 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-57cedd0d-e2fc-4761-bc47-5d6d65273f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:45.144504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:45.187444 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 703/1405] 10.222.0.17 () {58 vars in 1039 bytes} [Thu Jun 16 16:04:45 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:45.197472 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186e4bd-3d62-47bf-b187-2006a4477e91 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:45.212860 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186e4bd-3d62-47bf-b187-2006a4477e91 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:45.217981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3cd9df-c9ab-4047-aa48-b62e0107954f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0082e37ce54f49f7837601349508c89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:45.218645 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3cd9df-c9ab-4047-aa48-b62e0107954f service swift] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:45.218932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3cd9df-c9ab-4047-aa48-b62e0107954f service swift] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:45.219454 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3cd9df-c9ab-4047-aa48-b62e0107954f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:45.230205 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3cd9df-c9ab-4047-aa48-b62e0107954f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:45.231830 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3cd9df-c9ab-4047-aa48-b62e0107954f service swift] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:45.242269 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 703/1406] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 16:04:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:04:46.542266 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c83ac-26bc-4250-bafb-6ca3abd6dfbb None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:46.542772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c83ac-26bc-4250-bafb-6ca3abd6dfbb None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:46.542772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c83ac-26bc-4250-bafb-6ca3abd6dfbb None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:46.544128 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 704/1407] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:46.549430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328e26d-f0c7-437e-9331-09371e4d1fe7 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:46.550364 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328e26d-f0c7-437e-9331-09371e4d1fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:46.550715 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328e26d-f0c7-437e-9331-09371e4d1fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:46.570734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-7328e26d-f0c7-437e-9331-09371e4d1fe7 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:46.572217 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-7328e26d-f0c7-437e-9331-09371e4d1fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:46.585066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 704/1408] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:04:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:46.627755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c83ac-26bc-4250-bafb-6ca3abd6dfbb None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:46.644622 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c83ac-26bc-4250-bafb-6ca3abd6dfbb None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:46.650138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37642e4-a546-4e20-9109-45955e008d5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:46.650831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37642e4-a546-4e20-9109-45955e008d5f service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:46.651234 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37642e4-a546-4e20-9109-45955e008d5f service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:46.651415 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37642e4-a546-4e20-9109-45955e008d5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:46.661157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37642e4-a546-4e20-9109-45955e008d5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:46.665090 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37642e4-a546-4e20-9109-45955e008d5f service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:46.677766 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 705/1409] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:04:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:04:56.563770 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=55 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:56.671505 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=21 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:05:09.363129 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942019e-a709-4a6c-8032-a866e1621de0 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:09.363769 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942019e-a709-4a6c-8032-a866e1621de0 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:09.363769 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942019e-a709-4a6c-8032-a866e1621de0 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:09.365119 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 705/1410] 10.222.0.17 () {54 vars in 918 bytes} [Thu Jun 16 16:05:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:09.370281 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e19f1-f440-4575-82ee-6fe9d1d8cf0c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:09.370758 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e19f1-f440-4575-82ee-6fe9d1d8cf0c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:09.370855 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e19f1-f440-4575-82ee-6fe9d1d8cf0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:09.391108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ca9e19f1-f440-4575-82ee-6fe9d1d8cf0c None None] MFA Rules not processed for user `36b63614297346e29ff276eab444fe5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:09.392363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ca9e19f1-f440-4575-82ee-6fe9d1d8cf0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:09.403037 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 706/1411] 10.222.0.17 () {58 vars in 1015 bytes} [Thu Jun 16 16:05:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:09.421850 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2005f5-8abe-4203-ae6d-8326a50f68ff None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:09.422183 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2005f5-8abe-4203-ae6d-8326a50f68ff None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:09.422463 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2005f5-8abe-4203-ae6d-8326a50f68ff None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:09.423844 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 706/1412] 10.222.0.17 () {54 vars in 918 bytes} [Thu Jun 16 16:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:09.429261 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a8110-ec30-4230-a6b7-6c4422f34597 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:09.429549 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a8110-ec30-4230-a6b7-6c4422f34597 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:09.429830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a8110-ec30-4230-a6b7-6c4422f34597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:09.449659 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-623a8110-ec30-4230-a6b7-6c4422f34597 None None] MFA Rules not processed for user `c3645b1a0be34c01ac87c734fcd85ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:09.450915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-623a8110-ec30-4230-a6b7-6c4422f34597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:09.461181 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 707/1413] 10.222.0.17 () {58 vars in 1015 bytes} [Thu Jun 16 16:05:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:12.211904 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77dd5db-b243-4c66-bbc7-3287df048f92 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:12.212541 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77dd5db-b243-4c66-bbc7-3287df048f92 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:12.212541 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77dd5db-b243-4c66-bbc7-3287df048f92 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:12.213885 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:12.219764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24905fa9-aa8f-469b-90ec-89da005221c9 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:12.220272 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24905fa9-aa8f-469b-90ec-89da005221c9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:12.220388 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24905fa9-aa8f-469b-90ec-89da005221c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:12.239964 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-24905fa9-aa8f-469b-90ec-89da005221c9 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:12.241188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-24905fa9-aa8f-469b-90ec-89da005221c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:12.249218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:05:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:12.294254 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77dd5db-b243-4c66-bbc7-3287df048f92 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:12.310756 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77dd5db-b243-4c66-bbc7-3287df048f92 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:12.316882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995cd388-f65b-4446-9367-3d8d698bb6e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:12.317558 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cd388-f65b-4446-9367-3d8d698bb6e9 service nova] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:12.317840 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cd388-f65b-4446-9367-3d8d698bb6e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:12.318145 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cd388-f65b-4446-9367-3d8d698bb6e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:12.327913 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995cd388-f65b-4446-9367-3d8d698bb6e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:12.329584 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995cd388-f65b-4446-9367-3d8d698bb6e9 service nova] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:12.340534 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 708/1416] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:05:13.407174 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be974bfb-7aee-40f4-bccf-5e5a36b723d9 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:13.407745 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be974bfb-7aee-40f4-bccf-5e5a36b723d9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:13.407745 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be974bfb-7aee-40f4-bccf-5e5a36b723d9 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:13.410111 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:05:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:13.415552 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecdb61e-af00-47b9-a161-fff52f151424 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:13.416070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecdb61e-af00-47b9-a161-fff52f151424 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:13.416174 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecdb61e-af00-47b9-a161-fff52f151424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:13.435604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-0ecdb61e-af00-47b9-a161-fff52f151424 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:13.436824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-0ecdb61e-af00-47b9-a161-fff52f151424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:13.445309 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:05:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:13.619076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be974bfb-7aee-40f4-bccf-5e5a36b723d9 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:13.634471 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be974bfb-7aee-40f4-bccf-5e5a36b723d9 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:13.640133 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d46c7e-6afa-43e2-a0a4-e793c3494dd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:13.640797 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d46c7e-6afa-43e2-a0a4-e793c3494dd6 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:13.641076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d46c7e-6afa-43e2-a0a4-e793c3494dd6 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:13.641399 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d46c7e-6afa-43e2-a0a4-e793c3494dd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:13.652008 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d46c7e-6afa-43e2-a0a4-e793c3494dd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:13.653763 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d46c7e-6afa-43e2-a0a4-e793c3494dd6 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:13.665184 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 710/1419] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:14.689040 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102c235-cb2a-4dac-a3be-213f61ab8b28 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:14.689679 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102c235-cb2a-4dac-a3be-213f61ab8b28 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:14.689679 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102c235-cb2a-4dac-a3be-213f61ab8b28 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:14.691683 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:05:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:14.697075 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a4ea1-41f6-4095-a4b7-18c7d84792da None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:14.697620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a4ea1-41f6-4095-a4b7-18c7d84792da None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:14.697773 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a4ea1-41f6-4095-a4b7-18c7d84792da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:14.717228 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e10a4ea1-41f6-4095-a4b7-18c7d84792da None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:14.718490 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e10a4ea1-41f6-4095-a4b7-18c7d84792da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:14.728625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:14.888270 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102c235-cb2a-4dac-a3be-213f61ab8b28 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:14.903952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102c235-cb2a-4dac-a3be-213f61ab8b28 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:14.910163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b5350-1449-4ad6-8aa6-29eb88d7d935 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:14.910860 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b5350-1449-4ad6-8aa6-29eb88d7d935 service nova] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:14.911140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b5350-1449-4ad6-8aa6-29eb88d7d935 service nova] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:14.911451 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b5350-1449-4ad6-8aa6-29eb88d7d935 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:14.921118 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b5350-1449-4ad6-8aa6-29eb88d7d935 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:14.922867 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b5350-1449-4ad6-8aa6-29eb88d7d935 service nova] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:14.934451 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 711/1422] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:15.929357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be9619-1ada-446d-b15a-4c30ced7b5c6 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:15.929902 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be9619-1ada-446d-b15a-4c30ced7b5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:15.929902 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be9619-1ada-446d-b15a-4c30ced7b5c6 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:15.931300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:15.936803 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d14cda-70bc-4071-9d04-dd591db46b47 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:15.937270 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d14cda-70bc-4071-9d04-dd591db46b47 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:15.937374 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d14cda-70bc-4071-9d04-dd591db46b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:15.956875 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-99d14cda-70bc-4071-9d04-dd591db46b47 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:15.958126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-99d14cda-70bc-4071-9d04-dd591db46b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:15.967180 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:05:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:16.125579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be9619-1ada-446d-b15a-4c30ced7b5c6 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:16.141670 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be9619-1ada-446d-b15a-4c30ced7b5c6 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:16.147506 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485d8b3-59ba-458d-b984-a232a47e88da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:16.148146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485d8b3-59ba-458d-b984-a232a47e88da service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:16.148422 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485d8b3-59ba-458d-b984-a232a47e88da service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:16.148717 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485d8b3-59ba-458d-b984-a232a47e88da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:16.156927 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485d8b3-59ba-458d-b984-a232a47e88da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:16.158566 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485d8b3-59ba-458d-b984-a232a47e88da service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:16.169508 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 713/1425] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:17.106129 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e80705-647e-4706-b323-fe73d79acc02 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:17.106725 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e80705-647e-4706-b323-fe73d79acc02 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:17.106725 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e80705-647e-4706-b323-fe73d79acc02 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:17.108933 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:05:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:17.114828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adc8cf-387c-45f8-8990-9f3b3859c8f8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:17.115368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adc8cf-387c-45f8-8990-9f3b3859c8f8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:17.115368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adc8cf-387c-45f8-8990-9f3b3859c8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:17.135314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-86adc8cf-387c-45f8-8990-9f3b3859c8f8 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:17.136531 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-86adc8cf-387c-45f8-8990-9f3b3859c8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:17.145420 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:05:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:17.302389 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e80705-647e-4706-b323-fe73d79acc02 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:17.317341 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e80705-647e-4706-b323-fe73d79acc02 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:17.322972 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86c6d5-c9d5-4371-a6f9-e574d704e8ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:17.323627 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86c6d5-c9d5-4371-a6f9-e574d704e8ed service nova] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:17.323914 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86c6d5-c9d5-4371-a6f9-e574d704e8ed service nova] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:17.324192 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86c6d5-c9d5-4371-a6f9-e574d704e8ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:17.333536 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c86c6d5-c9d5-4371-a6f9-e574d704e8ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:17.335197 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c86c6d5-c9d5-4371-a6f9-e574d704e8ed service nova] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:17.345347 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 714/1428] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:18.230730 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b5334-6621-48b3-adaa-5476867a23b8 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:18.231290 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b5334-6621-48b3-adaa-5476867a23b8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:18.231290 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b5334-6621-48b3-adaa-5476867a23b8 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:18.232704 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:18.239080 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce859f-4736-4f41-aa1f-adf745cd196e None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:18.239607 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce859f-4736-4f41-aa1f-adf745cd196e None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:18.239727 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce859f-4736-4f41-aa1f-adf745cd196e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:18.258628 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-a9ce859f-4736-4f41-aa1f-adf745cd196e None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:18.260126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-a9ce859f-4736-4f41-aa1f-adf745cd196e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:18.268702 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:05:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:05:18.421628 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b5334-6621-48b3-adaa-5476867a23b8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:18.436692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b5334-6621-48b3-adaa-5476867a23b8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:18.443153 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd27faaf-7686-48b7-9a25-0b49a29b9179 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:18.443795 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27faaf-7686-48b7-9a25-0b49a29b9179 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:18.444081 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27faaf-7686-48b7-9a25-0b49a29b9179 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:18.444355 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27faaf-7686-48b7-9a25-0b49a29b9179 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:18.452692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd27faaf-7686-48b7-9a25-0b49a29b9179 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:18.454310 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd27faaf-7686-48b7-9a25-0b49a29b9179 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:18.463874 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 716/1431] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:19.363899 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6a968-20ac-4e02-a14c-865a9ecdc44c None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:19.364485 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6a968-20ac-4e02-a14c-865a9ecdc44c None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:19.364485 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6a968-20ac-4e02-a14c-865a9ecdc44c None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:19.366225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:05:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:19.372529 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0405a-f9da-4b07-b0d4-3fdb1033c258 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:19.373174 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0405a-f9da-4b07-b0d4-3fdb1033c258 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:19.373295 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0405a-f9da-4b07-b0d4-3fdb1033c258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:19.396472 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-57c0405a-f9da-4b07-b0d4-3fdb1033c258 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:19.397717 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-57c0405a-f9da-4b07-b0d4-3fdb1033c258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:19.406216 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:05:19.579459 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf6a968-20ac-4e02-a14c-865a9ecdc44c None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:19.594917 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf6a968-20ac-4e02-a14c-865a9ecdc44c None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:19.600770 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0a39e-e95c-4b41-8ddc-2c04f345f707 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:19.601424 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0a39e-e95c-4b41-8ddc-2c04f345f707 service nova] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:19.601708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0a39e-e95c-4b41-8ddc-2c04f345f707 service nova] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:19.602050 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0a39e-e95c-4b41-8ddc-2c04f345f707 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:19.611163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d0a39e-e95c-4b41-8ddc-2c04f345f707 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:19.612934 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d0a39e-e95c-4b41-8ddc-2c04f345f707 service nova] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:19.623556 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 717/1434] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:20.520252 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8228d99-d7eb-47f8-a609-6db7acb36f29 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:20.520790 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8228d99-d7eb-47f8-a609-6db7acb36f29 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:20.520790 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8228d99-d7eb-47f8-a609-6db7acb36f29 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:20.522138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:20.527298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcfe4c-58ae-4277-80f4-3358831aba14 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:20.527768 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcfe4c-58ae-4277-80f4-3358831aba14 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:20.527860 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcfe4c-58ae-4277-80f4-3358831aba14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:20.546297 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-a4fcfe4c-58ae-4277-80f4-3358831aba14 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:20.547532 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-a4fcfe4c-58ae-4277-80f4-3358831aba14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:20.556255 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:20.711887 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8228d99-d7eb-47f8-a609-6db7acb36f29 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:20.726914 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8228d99-d7eb-47f8-a609-6db7acb36f29 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:20.732275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355d6d5-3c48-46f5-a250-1e6b379f0c8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:20.732913 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355d6d5-3c48-46f5-a250-1e6b379f0c8e service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:20.733195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355d6d5-3c48-46f5-a250-1e6b379f0c8e service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:20.733470 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355d6d5-3c48-46f5-a250-1e6b379f0c8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:20.742556 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355d6d5-3c48-46f5-a250-1e6b379f0c8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:20.744155 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355d6d5-3c48-46f5-a250-1e6b379f0c8e service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:20.754344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 719/1437] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:21.630137 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc57039-5819-45c2-af6a-2646e2af387a None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:21.630725 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc57039-5819-45c2-af6a-2646e2af387a None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:21.630725 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc57039-5819-45c2-af6a-2646e2af387a None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:21.632089 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:21.637423 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb27cb-e21e-4d81-ae3a-9ced69c6254b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:21.637831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb27cb-e21e-4d81-ae3a-9ced69c6254b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:21.638042 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb27cb-e21e-4d81-ae3a-9ced69c6254b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:21.656626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-bffb27cb-e21e-4d81-ae3a-9ced69c6254b None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:21.657833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bffb27cb-e21e-4d81-ae3a-9ced69c6254b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:21.666834 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:05:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:21.832418 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc57039-5819-45c2-af6a-2646e2af387a None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:21.847848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc57039-5819-45c2-af6a-2646e2af387a None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:21.853480 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05222627-fc8b-4bbb-b5f6-257358b586d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:21.854165 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05222627-fc8b-4bbb-b5f6-257358b586d9 service nova] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:21.854453 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05222627-fc8b-4bbb-b5f6-257358b586d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:21.854729 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05222627-fc8b-4bbb-b5f6-257358b586d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:21.864893 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05222627-fc8b-4bbb-b5f6-257358b586d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:21.866611 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05222627-fc8b-4bbb-b5f6-257358b586d9 service nova] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:21.878183 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 720/1440] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:23.955970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693c4df-1767-4621-aaa6-67072c997e20 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:23.956551 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693c4df-1767-4621-aaa6-67072c997e20 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:23.956551 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693c4df-1767-4621-aaa6-67072c997e20 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:23.957968 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:23.963498 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594af72d-649b-4873-98f6-a14c8920081a None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:23.963927 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594af72d-649b-4873-98f6-a14c8920081a None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:23.964065 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594af72d-649b-4873-98f6-a14c8920081a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:23.983218 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-594af72d-649b-4873-98f6-a14c8920081a None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:23.984413 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-594af72d-649b-4873-98f6-a14c8920081a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:23.993198 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:05:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:24.031296 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693c4df-1767-4621-aaa6-67072c997e20 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:24.043718 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693c4df-1767-4621-aaa6-67072c997e20 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:24.047873 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95088fe0-d49e-4a1a-a425-88c106c32218 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:24.048025 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95088fe0-d49e-4a1a-a425-88c106c32218 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:24.048266 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95088fe0-d49e-4a1a-a425-88c106c32218 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:24.048544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95088fe0-d49e-4a1a-a425-88c106c32218 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:24.058612 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95088fe0-d49e-4a1a-a425-88c106c32218 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:24.060051 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95088fe0-d49e-4a1a-a425-88c106c32218 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:24.069940 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 722/1443] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:26.450370 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eea817-6ab3-4b6d-98b0-cceee8f9ec76 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:26.450934 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eea817-6ab3-4b6d-98b0-cceee8f9ec76 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:26.450934 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eea817-6ab3-4b6d-98b0-cceee8f9ec76 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:26.452315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:26.458120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f5d98-ff42-419a-9a05-c1cc02e961c0 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:26.458511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f5d98-ff42-419a-9a05-c1cc02e961c0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:26.458670 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f5d98-ff42-419a-9a05-c1cc02e961c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:26.476724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-837f5d98-ff42-419a-9a05-c1cc02e961c0 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:26.477838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-837f5d98-ff42-419a-9a05-c1cc02e961c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:26.487908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:05:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:26.526679 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eea817-6ab3-4b6d-98b0-cceee8f9ec76 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:26.538534 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eea817-6ab3-4b6d-98b0-cceee8f9ec76 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:26.542137 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d5e3b-6e39-4bfb-9f99-88da0ea52ec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:26.542782 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d5e3b-6e39-4bfb-9f99-88da0ea52ec3 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:26.543067 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d5e3b-6e39-4bfb-9f99-88da0ea52ec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:26.543345 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d5e3b-6e39-4bfb-9f99-88da0ea52ec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:26.552716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881d5e3b-6e39-4bfb-9f99-88da0ea52ec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:26.554196 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881d5e3b-6e39-4bfb-9f99-88da0ea52ec3 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:26.563568 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 723/1446] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:29.180653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e9dea-7737-4186-a20e-68e0874f7a6a None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:29.181265 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e9dea-7737-4186-a20e-68e0874f7a6a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:29.181265 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e9dea-7737-4186-a20e-68e0874f7a6a None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:29.182611 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:29.188203 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b8570-b11f-4afc-a6cf-13d88af23859 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:29.188639 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b8570-b11f-4afc-a6cf-13d88af23859 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:29.188800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b8570-b11f-4afc-a6cf-13d88af23859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:29.207128 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-203b8570-b11f-4afc-a6cf-13d88af23859 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:29.208643 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-203b8570-b11f-4afc-a6cf-13d88af23859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:29.216791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:05:29.257058 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e9dea-7737-4186-a20e-68e0874f7a6a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:29.268913 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e9dea-7737-4186-a20e-68e0874f7a6a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:29.272985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce2a36-374b-4e07-8106-57bbabdb918f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:29.273638 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce2a36-374b-4e07-8106-57bbabdb918f service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:29.273945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce2a36-374b-4e07-8106-57bbabdb918f service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:29.274233 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce2a36-374b-4e07-8106-57bbabdb918f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:29.284906 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce2a36-374b-4e07-8106-57bbabdb918f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:29.286366 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce2a36-374b-4e07-8106-57bbabdb918f service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:29.295801 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 725/1449] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:31.620419 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44ffe2-8b78-4a53-badc-40d424c6247e None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:31.621003 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44ffe2-8b78-4a53-badc-40d424c6247e None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:31.621099 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44ffe2-8b78-4a53-badc-40d424c6247e None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:31.622486 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:05:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:31.628675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a8e8f-042a-4ee7-8f5e-93aba7f7bdc1 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:31.629123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a8e8f-042a-4ee7-8f5e-93aba7f7bdc1 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:31.629228 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a8e8f-042a-4ee7-8f5e-93aba7f7bdc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:31.647844 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-a16a8e8f-042a-4ee7-8f5e-93aba7f7bdc1 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:31.649047 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a16a8e8f-042a-4ee7-8f5e-93aba7f7bdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:31.656888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:05:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:31.695600 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44ffe2-8b78-4a53-badc-40d424c6247e None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:31.707260 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44ffe2-8b78-4a53-badc-40d424c6247e None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:31.710774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebf150-1b4d-4440-8b44-dd3da16db1af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:31.711406 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebf150-1b4d-4440-8b44-dd3da16db1af service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:31.711686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebf150-1b4d-4440-8b44-dd3da16db1af service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:31.711960 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebf150-1b4d-4440-8b44-dd3da16db1af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:31.721380 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bebf150-1b4d-4440-8b44-dd3da16db1af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:31.722853 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bebf150-1b4d-4440-8b44-dd3da16db1af service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:31.732160 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 726/1452] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:33.964325 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cafcdc-914c-4c82-a83a-51e45bdcf144 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:33.964879 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cafcdc-914c-4c82-a83a-51e45bdcf144 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:33.964879 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cafcdc-914c-4c82-a83a-51e45bdcf144 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:33.966295 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:33.971471 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1c720-14db-428a-b8c2-d825e805362b None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:33.971915 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1c720-14db-428a-b8c2-d825e805362b None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:33.972032 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1c720-14db-428a-b8c2-d825e805362b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:33.991536 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-43e1c720-14db-428a-b8c2-d825e805362b None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:33.992752 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-43e1c720-14db-428a-b8c2-d825e805362b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:34.000710 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:05:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:34.039066 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cafcdc-914c-4c82-a83a-51e45bdcf144 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:34.050512 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cafcdc-914c-4c82-a83a-51e45bdcf144 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:34.054253 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d8795-3985-4724-91a0-05e1e0d14bfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:34.054891 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d8795-3985-4724-91a0-05e1e0d14bfa service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:34.055180 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d8795-3985-4724-91a0-05e1e0d14bfa service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:34.055455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d8795-3985-4724-91a0-05e1e0d14bfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:34.065429 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d8795-3985-4724-91a0-05e1e0d14bfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:34.066924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d8795-3985-4724-91a0-05e1e0d14bfa service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:34.076297 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 728/1455] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:36.390767 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1c139-bfe9-40a0-bd3b-3a795fd77b5a None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:36.391339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1c139-bfe9-40a0-bd3b-3a795fd77b5a None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:36.391339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1c139-bfe9-40a0-bd3b-3a795fd77b5a None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:36.392856 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:36.398172 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d7f01-4ea0-4990-89d1-76a94ffcf5bd None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:36.398653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d7f01-4ea0-4990-89d1-76a94ffcf5bd None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:36.398764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d7f01-4ea0-4990-89d1-76a94ffcf5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:36.417508 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b64d7f01-4ea0-4990-89d1-76a94ffcf5bd None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:36.418768 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b64d7f01-4ea0-4990-89d1-76a94ffcf5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:36.427384 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:05:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:36.464922 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1c139-bfe9-40a0-bd3b-3a795fd77b5a None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:36.476115 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1c139-bfe9-40a0-bd3b-3a795fd77b5a None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:36.479933 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162979d7-7c36-4db4-a74e-440834eb27fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:36.480559 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162979d7-7c36-4db4-a74e-440834eb27fb service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:36.480863 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162979d7-7c36-4db4-a74e-440834eb27fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:36.481143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162979d7-7c36-4db4-a74e-440834eb27fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:36.491457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162979d7-7c36-4db4-a74e-440834eb27fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:36.492898 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162979d7-7c36-4db4-a74e-440834eb27fb service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:36.502979 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 729/1458] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:38.741250 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150a688-3660-4d7c-bdd0-3c94685e4274 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:38.741828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150a688-3660-4d7c-bdd0-3c94685e4274 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:38.741828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150a688-3660-4d7c-bdd0-3c94685e4274 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:38.743241 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:38.748824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526dfe75-f832-47d7-81c7-dcb7adc15c95 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:38.749277 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526dfe75-f832-47d7-81c7-dcb7adc15c95 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:38.749381 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526dfe75-f832-47d7-81c7-dcb7adc15c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:38.755646 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=70 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:05:38.769436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-526dfe75-f832-47d7-81c7-dcb7adc15c95 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:38.770674 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-526dfe75-f832-47d7-81c7-dcb7adc15c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:38.778883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:05:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:38.816553 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150a688-3660-4d7c-bdd0-3c94685e4274 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:38.821993 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=73 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:05:38.827967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150a688-3660-4d7c-bdd0-3c94685e4274 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:38.832374 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64cf9bf-082f-4a49-a849-b431bfd38a9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:38.833006 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cf9bf-082f-4a49-a849-b431bfd38a9e service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:38.833287 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cf9bf-082f-4a49-a849-b431bfd38a9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:38.833558 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cf9bf-082f-4a49-a849-b431bfd38a9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:38.842473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64cf9bf-082f-4a49-a849-b431bfd38a9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:38.843922 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64cf9bf-082f-4a49-a849-b431bfd38a9e service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:38.853968 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 731/1461] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:41.075190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed58db3-edc1-445a-afe6-2338bb3edaaf None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:41.075748 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed58db3-edc1-445a-afe6-2338bb3edaaf None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:41.075748 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed58db3-edc1-445a-afe6-2338bb3edaaf None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:41.077135 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:41.082508 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b868f6d-3e72-4d74-adb3-dbc87d294c8a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:41.082908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b868f6d-3e72-4d74-adb3-dbc87d294c8a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:41.083063 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b868f6d-3e72-4d74-adb3-dbc87d294c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:41.101295 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8b868f6d-3e72-4d74-adb3-dbc87d294c8a None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:41.102546 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8b868f6d-3e72-4d74-adb3-dbc87d294c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:41.110532 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:05:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:41.148763 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed58db3-edc1-445a-afe6-2338bb3edaaf None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:41.160306 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed58db3-edc1-445a-afe6-2338bb3edaaf None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:41.164042 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4f9ea-e314-4f85-a83b-9eafd994aec6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:41.164692 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4f9ea-e314-4f85-a83b-9eafd994aec6 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:41.164977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4f9ea-e314-4f85-a83b-9eafd994aec6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:41.165260 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4f9ea-e314-4f85-a83b-9eafd994aec6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:41.175168 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4f9ea-e314-4f85-a83b-9eafd994aec6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:41.176605 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4f9ea-e314-4f85-a83b-9eafd994aec6 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:41.186166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 732/1464] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:43.406258 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e743a-31aa-44c6-855b-867c06a04f9f None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:43.406832 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e743a-31aa-44c6-855b-867c06a04f9f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:43.406832 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e743a-31aa-44c6-855b-867c06a04f9f None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:43.408747 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:05:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:43.414018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37496bf-0a30-4cda-be43-e129d34381af None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:43.414476 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37496bf-0a30-4cda-be43-e129d34381af None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:43.414575 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37496bf-0a30-4cda-be43-e129d34381af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:43.432540 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-c37496bf-0a30-4cda-be43-e129d34381af None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:43.433749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-c37496bf-0a30-4cda-be43-e129d34381af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:43.442301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:05:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:43.480607 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e743a-31aa-44c6-855b-867c06a04f9f None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:43.492319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e743a-31aa-44c6-855b-867c06a04f9f None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:43.496018 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f6c76-4653-4506-9154-514bf350bf0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:43.496665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f6c76-4653-4506-9154-514bf350bf0c service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:43.496949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f6c76-4653-4506-9154-514bf350bf0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:43.497257 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f6c76-4653-4506-9154-514bf350bf0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:43.508007 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f6c76-4653-4506-9154-514bf350bf0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:43.509693 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f6c76-4653-4506-9154-514bf350bf0c service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:43.519930 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 734/1467] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:45.692062 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff2fbc-6b03-484d-88e5-a9b6da6827aa None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:45.692683 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff2fbc-6b03-484d-88e5-a9b6da6827aa None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:45.692683 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff2fbc-6b03-484d-88e5-a9b6da6827aa None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:45.694047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:45.699639 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c60b9-3f92-4033-947c-265f3d1060a5 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:45.700153 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c60b9-3f92-4033-947c-265f3d1060a5 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:45.700258 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c60b9-3f92-4033-947c-265f3d1060a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:45.717916 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0f1c60b9-3f92-4033-947c-265f3d1060a5 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:45.719158 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0f1c60b9-3f92-4033-947c-265f3d1060a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:45.727463 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:05:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:45.764902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff2fbc-6b03-484d-88e5-a9b6da6827aa None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:45.776209 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff2fbc-6b03-484d-88e5-a9b6da6827aa None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:45.779808 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ccdbc-df68-4c03-bc3b-14c18bd75fe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:45.780450 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ccdbc-df68-4c03-bc3b-14c18bd75fe3 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:45.780730 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ccdbc-df68-4c03-bc3b-14c18bd75fe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:45.781007 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ccdbc-df68-4c03-bc3b-14c18bd75fe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:45.791400 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539ccdbc-df68-4c03-bc3b-14c18bd75fe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:45.792852 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539ccdbc-df68-4c03-bc3b-14c18bd75fe3 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:45.802878 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 735/1470] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:05:55.712842 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=17 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:05:55.797998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=17 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:06:06.070702 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0603569-3b0f-40b0-ae7a-c1873b311464 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:06.071381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0603569-3b0f-40b0-ae7a-c1873b311464 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:06.071381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0603569-3b0f-40b0-ae7a-c1873b311464 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:06.072656 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 736/1471] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:06.077958 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccfc1d-c04b-491a-9498-a1ecd302c1e5 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:06.078375 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccfc1d-c04b-491a-9498-a1ecd302c1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:06.078512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccfc1d-c04b-491a-9498-a1ecd302c1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:06.096713 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-f2ccfc1d-c04b-491a-9498-a1ecd302c1e5 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:06.097955 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-f2ccfc1d-c04b-491a-9498-a1ecd302c1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:06.105859 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 736/1472] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:06:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:06.155163 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369787d-a5dc-4b27-b59d-9f66cc1108af None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:06.155445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369787d-a5dc-4b27-b59d-9f66cc1108af None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:06.155722 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369787d-a5dc-4b27-b59d-9f66cc1108af None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:06.157083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 737/1473] 10.222.0.17 () {54 vars in 962 bytes} [Thu Jun 16 16:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:06.163088 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de97050-5847-4639-a906-2334240750ab None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:06.163378 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de97050-5847-4639-a906-2334240750ab None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:06.163657 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de97050-5847-4639-a906-2334240750ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:06.182148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-3de97050-5847-4639-a906-2334240750ab None None] MFA Rules not processed for user `6f11beca9c5148eea4cc45f8dc89dbda`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:06.183359 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-3de97050-5847-4639-a906-2334240750ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:06.190286 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 737/1474] 10.222.0.17 () {58 vars in 1059 bytes} [Thu Jun 16 16:06:06 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:06.197809 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369787d-a5dc-4b27-b59d-9f66cc1108af None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:06.208584 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369787d-a5dc-4b27-b59d-9f66cc1108af None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:06.212125 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd53ce-e439-4248-88ba-b5977aeec2b1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f11beca9c5148eea4cc45f8dc89dbda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:06.212756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd53ce-e439-4248-88ba-b5977aeec2b1 service glance] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:06.213043 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd53ce-e439-4248-88ba-b5977aeec2b1 service glance] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:06.213323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd53ce-e439-4248-88ba-b5977aeec2b1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:06.222926 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd53ce-e439-4248-88ba-b5977aeec2b1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:06.224356 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd53ce-e439-4248-88ba-b5977aeec2b1 service glance] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:06.233718 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 738/1475] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:06:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:06:08.503819 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11315221-1484-46f8-83a1-23e117786309 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:08.504382 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11315221-1484-46f8-83a1-23e117786309 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:08.504382 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11315221-1484-46f8-83a1-23e117786309 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:08.505776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 738/1476] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:08.511429 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320c5ea-508a-413d-b12d-e60be49131fb None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:08.511872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320c5ea-508a-413d-b12d-e60be49131fb None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:08.511977 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320c5ea-508a-413d-b12d-e60be49131fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:08.538818 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-a320c5ea-508a-413d-b12d-e60be49131fb None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:08.540034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a320c5ea-508a-413d-b12d-e60be49131fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:08.547945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 739/1477] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:06:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:08.571607 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11315221-1484-46f8-83a1-23e117786309 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:08.585361 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11315221-1484-46f8-83a1-23e117786309 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:08.591435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781cd6ba-634d-4391-b9ba-fa2c6d71cf43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:08.592100 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781cd6ba-634d-4391-b9ba-fa2c6d71cf43 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:08.592381 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781cd6ba-634d-4391-b9ba-fa2c6d71cf43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:08.592656 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781cd6ba-634d-4391-b9ba-fa2c6d71cf43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:08.593401 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781cd6ba-634d-4391-b9ba-fa2c6d71cf43 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:08.595874 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: WARNING py.warnings [None req-781cd6ba-634d-4391-b9ba-fa2c6d71cf43 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:06:08.595874 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: warnings.warn(msg) Jun 16 16:06:08.595874 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: Jun 16 16:06:08.597037 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781cd6ba-634d-4391-b9ba-fa2c6d71cf43 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:08.617703 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 739/1478] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 16:06:08 2022] GET /identity/v3/projects => generated 2553 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:06:10.823473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182403fd-0c30-4153-b68e-5b943afb6b19 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:10.824060 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182403fd-0c30-4153-b68e-5b943afb6b19 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:10.824060 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182403fd-0c30-4153-b68e-5b943afb6b19 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:10.825401 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 740/1479] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:10.831366 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95bf2b-c37d-4403-8882-684540814734 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:10.831848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95bf2b-c37d-4403-8882-684540814734 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:10.831952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95bf2b-c37d-4403-8882-684540814734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:10.850481 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-ca95bf2b-c37d-4403-8882-684540814734 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:10.851691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-ca95bf2b-c37d-4403-8882-684540814734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:10.859760 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 740/1480] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:10.903069 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182403fd-0c30-4153-b68e-5b943afb6b19 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:10.918137 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182403fd-0c30-4153-b68e-5b943afb6b19 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:10.924059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21b2df-09e6-4024-8a5d-de976677bf34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:10.924694 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21b2df-09e6-4024-8a5d-de976677bf34 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:10.924970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21b2df-09e6-4024-8a5d-de976677bf34 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:10.925243 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21b2df-09e6-4024-8a5d-de976677bf34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:10.933546 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa21b2df-09e6-4024-8a5d-de976677bf34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:10.935186 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa21b2df-09e6-4024-8a5d-de976677bf34 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:10.945345 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 741/1481] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:06:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:06:13.282186 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ab242-bb9e-45bc-a901-7ff081b4451f None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:13.282922 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ab242-bb9e-45bc-a901-7ff081b4451f None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:13.282922 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ab242-bb9e-45bc-a901-7ff081b4451f None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:13.284125 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 741/1482] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:13.289703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af72ed-39d9-498b-b4d0-cf29c84088aa None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:13.290288 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af72ed-39d9-498b-b4d0-cf29c84088aa None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:13.290414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af72ed-39d9-498b-b4d0-cf29c84088aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:13.308505 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-93af72ed-39d9-498b-b4d0-cf29c84088aa None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:13.309839 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-93af72ed-39d9-498b-b4d0-cf29c84088aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:13.318455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 742/1483] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:06:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:13.354227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ab242-bb9e-45bc-a901-7ff081b4451f None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:13.365630 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ab242-bb9e-45bc-a901-7ff081b4451f None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:13.369147 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2d776-f8fa-430f-8374-0c0eda0d67eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:13.369783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2d776-f8fa-430f-8374-0c0eda0d67eb service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:13.370101 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2d776-f8fa-430f-8374-0c0eda0d67eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:13.370384 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2d776-f8fa-430f-8374-0c0eda0d67eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:13.379827 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b2d776-f8fa-430f-8374-0c0eda0d67eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:13.381270 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b2d776-f8fa-430f-8374-0c0eda0d67eb service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:13.391084 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 742/1484] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:06:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:06:15.579594 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c20023-2062-4ac5-9d42-6d0028b226eb None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:15.580170 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c20023-2062-4ac5-9d42-6d0028b226eb None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:15.580297 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c20023-2062-4ac5-9d42-6d0028b226eb None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:15.581711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 743/1485] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:06:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:06:15.587193 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c02cd-170f-48ad-88c9-684e76ba6a08 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:15.587619 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c02cd-170f-48ad-88c9-684e76ba6a08 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:15.587764 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c02cd-170f-48ad-88c9-684e76ba6a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:15.606310 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-223c02cd-170f-48ad-88c9-684e76ba6a08 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:15.607552 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-223c02cd-170f-48ad-88c9-684e76ba6a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:15.616137 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 743/1486] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:06:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:15.656472 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c20023-2062-4ac5-9d42-6d0028b226eb None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:15.667498 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c20023-2062-4ac5-9d42-6d0028b226eb None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:15.670964 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cffd28d-c9a0-4b2c-8445-e0d81479102c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:15.671593 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cffd28d-c9a0-4b2c-8445-e0d81479102c service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:15.671874 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cffd28d-c9a0-4b2c-8445-e0d81479102c service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:15.672147 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cffd28d-c9a0-4b2c-8445-e0d81479102c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:15.682118 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cffd28d-c9a0-4b2c-8445-e0d81479102c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:15.683568 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cffd28d-c9a0-4b2c-8445-e0d81479102c service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:15.693855 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 744/1487] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:06:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:06:17.993357 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffdb32-25d5-4187-aafa-b2abf04c15f5 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:17.993903 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffdb32-25d5-4187-aafa-b2abf04c15f5 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:17.993903 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffdb32-25d5-4187-aafa-b2abf04c15f5 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:17.995340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 744/1488] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:06:18.000724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0d8b2-3c43-44f3-82d3-ad0c039b550c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:18.002136 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0d8b2-3c43-44f3-82d3-ad0c039b550c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:18.002430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0d8b2-3c43-44f3-82d3-ad0c039b550c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:18.022122 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e8e0d8b2-3c43-44f3-82d3-ad0c039b550c None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:18.023318 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e8e0d8b2-3c43-44f3-82d3-ad0c039b550c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:18.031103 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 745/1489] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:06:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:18.071649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffdb32-25d5-4187-aafa-b2abf04c15f5 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:18.083102 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffdb32-25d5-4187-aafa-b2abf04c15f5 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:18.086674 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122c672-0407-444a-9074-e074aec70af5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:18.087312 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122c672-0407-444a-9074-e074aec70af5 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:18.087597 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122c672-0407-444a-9074-e074aec70af5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:18.087871 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122c672-0407-444a-9074-e074aec70af5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:18.097564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122c672-0407-444a-9074-e074aec70af5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:18.098993 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122c672-0407-444a-9074-e074aec70af5 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:18.108772 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 745/1490] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:06:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:06:20.541447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b3b57-cb0c-4f10-a3fe-9804e75c0c11 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:20.542053 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b3b57-cb0c-4f10-a3fe-9804e75c0c11 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:20.542053 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b3b57-cb0c-4f10-a3fe-9804e75c0c11 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:20.543408 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 746/1491] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 16:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:06:20.548961 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d4a19-28b0-4b04-856d-e78c0c927403 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:20.549379 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d4a19-28b0-4b04-856d-e78c0c927403 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:20.549522 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d4a19-28b0-4b04-856d-e78c0c927403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:20.567688 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-220d4a19-28b0-4b04-856d-e78c0c927403 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:20.568898 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-220d4a19-28b0-4b04-856d-e78c0c927403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:20.576875 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 746/1492] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 16:06:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:20.617688 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b3b57-cb0c-4f10-a3fe-9804e75c0c11 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:20.630078 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b3b57-cb0c-4f10-a3fe-9804e75c0c11 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:20.634267 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb12e60-dc6a-4ed9-9ff5-e5dd6a7e3a1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:20.635260 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb12e60-dc6a-4ed9-9ff5-e5dd6a7e3a1b service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:20.635490 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb12e60-dc6a-4ed9-9ff5-e5dd6a7e3a1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:20.635730 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb12e60-dc6a-4ed9-9ff5-e5dd6a7e3a1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:20.646159 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb12e60-dc6a-4ed9-9ff5-e5dd6a7e3a1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:20.647598 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb12e60-dc6a-4ed9-9ff5-e5dd6a7e3a1b service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:20.656804 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 747/1493] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:06:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:06:30.562031 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=33 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:06:30.652833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=33 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:07:48.423592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7cd19e-88f5-439a-8a66-708f04c8b8f4 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:07:48.424313 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7cd19e-88f5-439a-8a66-708f04c8b8f4 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:07:48.424313 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7cd19e-88f5-439a-8a66-708f04c8b8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:07:48.443646 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-fc7cd19e-88f5-439a-8a66-708f04c8b8f4 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:07:48.444862 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-fc7cd19e-88f5-439a-8a66-708f04c8b8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:07:48.453308 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 747/1494] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:07:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:07:48.463548 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf974a-4669-41a0-aa6a-be67558e89c5 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:07:48.464127 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf974a-4669-41a0-aa6a-be67558e89c5 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:07:48.464127 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf974a-4669-41a0-aa6a-be67558e89c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:07:48.483283 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c4cf974a-4669-41a0-aa6a-be67558e89c5 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:07:48.484491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c4cf974a-4669-41a0-aa6a-be67558e89c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:07:48.493618 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 748/1495] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:07:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:07:48.498956 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7cd19e-88f5-439a-8a66-708f04c8b8f4 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:07:48.510131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7cd19e-88f5-439a-8a66-708f04c8b8f4 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:07:48.515657 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1520771a-63ed-4e13-8d63-b8c280316bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:07:48.516491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1520771a-63ed-4e13-8d63-b8c280316bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:07:48.517508 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1520771a-63ed-4e13-8d63-b8c280316bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:07:48.518453 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1520771a-63ed-4e13-8d63-b8c280316bf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:07:48.519248 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1520771a-63ed-4e13-8d63-b8c280316bf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:07:48.519561 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1520771a-63ed-4e13-8d63-b8c280316bf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:07:48.520592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: WARNING py.warnings [None req-1520771a-63ed-4e13-8d63-b8c280316bf9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:07:48.520592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: warnings.warn(msg) Jun 16 16:07:48.520592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: Jun 16 16:07:48.521511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1520771a-63ed-4e13-8d63-b8c280316bf9 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:07:48.533616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 748/1496] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:07:48.541596 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf974a-4669-41a0-aa6a-be67558e89c5 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:07:48.551197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf974a-4669-41a0-aa6a-be67558e89c5 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:07:48.555843 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564932e-cbbb-46cb-8d41-c56b298d201d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:07:48.556495 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564932e-cbbb-46cb-8d41-c56b298d201d admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:07:48.556780 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564932e-cbbb-46cb-8d41-c56b298d201d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:07:48.557059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564932e-cbbb-46cb-8d41-c56b298d201d admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:07:48.557752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9564932e-cbbb-46cb-8d41-c56b298d201d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:07:48.559781 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: WARNING py.warnings [None req-9564932e-cbbb-46cb-8d41-c56b298d201d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:07:48.559781 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: warnings.warn(msg) Jun 16 16:07:48.559781 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: Jun 16 16:07:48.560992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9564932e-cbbb-46cb-8d41-c56b298d201d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:07:48.599150 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 749/1497] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:07:48 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:07:48.603517 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1520771a-63ed-4e13-8d63-b8c280316bf9 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:07:48.614273 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1520771a-63ed-4e13-8d63-b8c280316bf9 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:07:48.669383 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb4d0f-16a0-4aa3-af46-1386769d4bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:07:48.670081 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb4d0f-16a0-4aa3-af46-1386769d4bb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:07:48.670366 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb4d0f-16a0-4aa3-af46-1386769d4bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:07:48.670642 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb4d0f-16a0-4aa3-af46-1386769d4bb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:07:48.671320 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb4d0f-16a0-4aa3-af46-1386769d4bb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:07:48.672340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: WARNING py.warnings [None req-addb4d0f-16a0-4aa3-af46-1386769d4bb0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:07:48.672340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: warnings.warn(msg) Jun 16 16:07:48.672340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: Jun 16 16:07:48.674241 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb4d0f-16a0-4aa3-af46-1386769d4bb0 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:07:48.694149 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 749/1498] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:07:48 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:07:48.698516 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564932e-cbbb-46cb-8d41-c56b298d201d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:07:48.709413 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564932e-cbbb-46cb-8d41-c56b298d201d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:07:48.715201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da43de4-855d-4d7e-bf57-4d464b4694b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:07:48.715959 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da43de4-855d-4d7e-bf57-4d464b4694b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:07:48.716228 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da43de4-855d-4d7e-bf57-4d464b4694b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:07:48.716507 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da43de4-855d-4d7e-bf57-4d464b4694b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:07:48.717305 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da43de4-855d-4d7e-bf57-4d464b4694b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:07:48.718404 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: WARNING py.warnings [None req-5da43de4-855d-4d7e-bf57-4d464b4694b3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:07:48.718404 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: warnings.warn(msg) Jun 16 16:07:48.718404 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: Jun 16 16:07:48.719300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da43de4-855d-4d7e-bf57-4d464b4694b3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:07:48.732635 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 750/1499] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:07:48.737597 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb4d0f-16a0-4aa3-af46-1386769d4bb0 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:07:48.747905 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb4d0f-16a0-4aa3-af46-1386769d4bb0 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:07:48.754808 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b712231-8c6b-49b0-9ce6-bd4925ca5bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:07:48.755475 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b712231-8c6b-49b0-9ce6-bd4925ca5bfb admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:07:48.755756 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b712231-8c6b-49b0-9ce6-bd4925ca5bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:07:48.756031 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b712231-8c6b-49b0-9ce6-bd4925ca5bfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:07:48.756853 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b712231-8c6b-49b0-9ce6-bd4925ca5bfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:07:48.757871 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: WARNING py.warnings [None req-2b712231-8c6b-49b0-9ce6-bd4925ca5bfb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:07:48.757871 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: warnings.warn(msg) Jun 16 16:07:48.757871 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: Jun 16 16:07:48.758809 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b712231-8c6b-49b0-9ce6-bd4925ca5bfb admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:07:48.773844 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 750/1500] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:07:48.778111 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da43de4-855d-4d7e-bf57-4d464b4694b3 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:07:48.788362 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da43de4-855d-4d7e-bf57-4d464b4694b3 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:07:48.793005 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e894d-b47b-42a8-a111-ca1e25a5f1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:07:48.793897 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e894d-b47b-42a8-a111-ca1e25a5f1c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:07:48.794214 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e894d-b47b-42a8-a111-ca1e25a5f1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:07:48.794494 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e894d-b47b-42a8-a111-ca1e25a5f1c6 admin admin] PATH_INFO: `/v3/projects/56a27c56498e48bf820e71f3bb6df507/users/09ce707ea7294341a1bb90148978a236/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:07:48.810267 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e894d-b47b-42a8-a111-ca1e25a5f1c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56a27c56498e48bf820e71f3bb6df507, user_id=09ce707ea7294341a1bb90148978a236, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:07:48.811329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: WARNING py.warnings [None req-f21e894d-b47b-42a8-a111-ca1e25a5f1c6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:07:48.811329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: warnings.warn(msg) Jun 16 16:07:48.811329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: Jun 16 16:07:48.813273 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e894d-b47b-42a8-a111-ca1e25a5f1c6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:07:48.827072 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 751/1501] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:07:48 2022] PUT /identity/v3/projects/56a27c56498e48bf820e71f3bb6df507/users/09ce707ea7294341a1bb90148978a236/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:07:48.838701 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc482c6-d488-4a8b-b5c9-3ce9b216155d None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:07:48.838996 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc482c6-d488-4a8b-b5c9-3ce9b216155d None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:07:48.839277 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc482c6-d488-4a8b-b5c9-3ce9b216155d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:07:48.867915 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-9fc482c6-d488-4a8b-b5c9-3ce9b216155d None None] MFA Rules not processed for user `09ce707ea7294341a1bb90148978a236`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:07:48.869127 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-9fc482c6-d488-4a8b-b5c9-3ce9b216155d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:07:48.936758 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 751/1502] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:07:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:07:48.951069 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df3961-0799-4e0c-ae46-d8ec74e137ac None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:07:48.951445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df3961-0799-4e0c-ae46-d8ec74e137ac None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:07:48.951733 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df3961-0799-4e0c-ae46-d8ec74e137ac None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:07:48.953143 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 752/1503] 10.222.0.17 () {54 vars in 963 bytes} [Thu Jun 16 16:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:07:48.958574 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35eb6d-c5b5-4cde-aa6f-bc9650766eed None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:07:48.958853 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35eb6d-c5b5-4cde-aa6f-bc9650766eed None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:07:48.959266 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35eb6d-c5b5-4cde-aa6f-bc9650766eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:07:48.978363 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-5d35eb6d-c5b5-4cde-aa6f-bc9650766eed None None] MFA Rules not processed for user `69b669a5ba4e4294baeea369949e3e6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:07:48.979597 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-5d35eb6d-c5b5-4cde-aa6f-bc9650766eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:07:49.027333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:07:49.036809 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 752/1504] 10.222.0.17 () {58 vars in 1060 bytes} [Thu Jun 16 16:07:48 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:07:49.044864 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df3961-0799-4e0c-ae46-d8ec74e137ac None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:07:49.055487 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df3961-0799-4e0c-ae46-d8ec74e137ac None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:07:49.059017 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49edbb29-5d50-4d0a-be37-13b061877051 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:07:49.059645 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edbb29-5d50-4d0a-be37-13b061877051 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:07:49.059926 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edbb29-5d50-4d0a-be37-13b061877051 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:07:49.060205 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edbb29-5d50-4d0a-be37-13b061877051 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:07:49.069089 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49edbb29-5d50-4d0a-be37-13b061877051 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:07:49.070576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49edbb29-5d50-4d0a-be37-13b061877051 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:07:49.080082 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 753/1505] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:07:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:07:49.123512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72d655-9f74-4a6c-8ba2-ccc5ecbd946a None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:07:49.123828 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72d655-9f74-4a6c-8ba2-ccc5ecbd946a None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:07:49.124101 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72d655-9f74-4a6c-8ba2-ccc5ecbd946a None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:07:49.125429 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 753/1506] 10.222.0.17 () {52 vars in 829 bytes} [Thu Jun 16 16:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:07:49.130581 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49edbb29-5d50-4d0a-be37-13b061877051 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:07:49.193601 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49edbb29-5d50-4d0a-be37-13b061877051 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:07:49.198164 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed4872-a963-4a77-8d8a-4c8e3649c4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:07:49.199059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed4872-a963-4a77-8d8a-4c8e3649c4a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:07:49.199328 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed4872-a963-4a77-8d8a-4c8e3649c4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:07:49.199608 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed4872-a963-4a77-8d8a-4c8e3649c4a6 admin admin] PATH_INFO: `/v3/users/09ce707ea7294341a1bb90148978a236` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:07:49.200843 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ed4872-a963-4a77-8d8a-4c8e3649c4a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09ce707ea7294341a1bb90148978a236)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:07:49.201864 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: WARNING py.warnings [None req-e4ed4872-a963-4a77-8d8a-4c8e3649c4a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:07:49.201864 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: warnings.warn(msg) Jun 16 16:07:49.201864 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: Jun 16 16:07:49.203133 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ed4872-a963-4a77-8d8a-4c8e3649c4a6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:07:49.274180 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e4ed4872-a963-4a77-8d8a-4c8e3649c4a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09ce707ea7294341a1bb90148978a236'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:07:49.294071 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e4ed4872-a963-4a77-8d8a-4c8e3649c4a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09ce707ea7294341a1bb90148978a236'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:07:49.304459 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e4ed4872-a963-4a77-8d8a-4c8e3649c4a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09ce707ea7294341a1bb90148978a236'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:07:49.313289 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:07:49.319017 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e4ed4872-a963-4a77-8d8a-4c8e3649c4a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09ce707ea7294341a1bb90148978a236'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:07:49.319787 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e4ed4872-a963-4a77-8d8a-4c8e3649c4a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09ce707ea7294341a1bb90148978a236'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:07:49.326826 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 754/1507] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:07:49 2022] DELETE /identity/v3/users/09ce707ea7294341a1bb90148978a236 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:07:49.331875 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a618a3f-8955-4a23-94e8-54af865be75c None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:07:49.332305 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a618a3f-8955-4a23-94e8-54af865be75c None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:07:49.332445 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a618a3f-8955-4a23-94e8-54af865be75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:07:49.350740 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-9a618a3f-8955-4a23-94e8-54af865be75c None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:07:49.352771 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-9a618a3f-8955-4a23-94e8-54af865be75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:07:49.415323 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 754/1508] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:07:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:07:49.425623 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed4872-a963-4a77-8d8a-4c8e3649c4a6 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:07:49.427257 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e4ed4872-a963-4a77-8d8a-4c8e3649c4a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:07:49.488559 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed4872-a963-4a77-8d8a-4c8e3649c4a6 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:07:49.492997 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0fa4b1-3635-41ef-88e5-f9c3ee350f61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:07:49.494054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fa4b1-3635-41ef-88e5-f9c3ee350f61 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:07:49.494343 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fa4b1-3635-41ef-88e5-f9c3ee350f61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:07:49.495034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fa4b1-3635-41ef-88e5-f9c3ee350f61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:07:49.507829 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0fa4b1-3635-41ef-88e5-f9c3ee350f61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:07:49.509330 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0fa4b1-3635-41ef-88e5-f9c3ee350f61 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:07:49.519689 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 755/1509] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:07:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:07:49.858023 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a618a3f-8955-4a23-94e8-54af865be75c None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:07:49.859600 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-9a618a3f-8955-4a23-94e8-54af865be75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:07:49.872734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a618a3f-8955-4a23-94e8-54af865be75c None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:07:49.878118 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be94fa4-afe7-449e-80a3-3d57a8bac26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:07:49.878989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be94fa4-afe7-449e-80a3-3d57a8bac26c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:07:49.879337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be94fa4-afe7-449e-80a3-3d57a8bac26c admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:07:49.880143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be94fa4-afe7-449e-80a3-3d57a8bac26c admin admin] PATH_INFO: `/v3/projects/56a27c56498e48bf820e71f3bb6df507` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:07:49.881743 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be94fa4-afe7-449e-80a3-3d57a8bac26c admin admin] RBAC: Authorizing `identity:delete_project(project_id=56a27c56498e48bf820e71f3bb6df507)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:07:49.882826 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: WARNING py.warnings [None req-5be94fa4-afe7-449e-80a3-3d57a8bac26c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:07:49.882826 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: warnings.warn(msg) Jun 16 16:07:49.882826 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: Jun 16 16:07:49.888649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be94fa4-afe7-449e-80a3-3d57a8bac26c admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:07:49.953371 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-5be94fa4-afe7-449e-80a3-3d57a8bac26c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56a27c56498e48bf820e71f3bb6df507', 'request_id': 'req-5be94fa4-afe7-449e-80a3-3d57a8bac26c'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:07:49.960246 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-5be94fa4-afe7-449e-80a3-3d57a8bac26c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56a27c56498e48bf820e71f3bb6df507', 'request_id': 'req-5be94fa4-afe7-449e-80a3-3d57a8bac26c'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:07:49.971103 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-5be94fa4-afe7-449e-80a3-3d57a8bac26c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56a27c56498e48bf820e71f3bb6df507', 'request_id': 'req-5be94fa4-afe7-449e-80a3-3d57a8bac26c'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:07:49.990642 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-5be94fa4-afe7-449e-80a3-3d57a8bac26c admin admin] The token cache is being invalidate because project 56a27c56498e48bf820e71f3bb6df507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:07:49.990935 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-5be94fa4-afe7-449e-80a3-3d57a8bac26c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:07:49.991755 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-5be94fa4-afe7-449e-80a3-3d57a8bac26c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:07:49.993369 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 755/1510] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:07:49 2022] DELETE /identity/v3/projects/56a27c56498e48bf820e71f3bb6df507 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:07:59.515035 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=29 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:07:59.987501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:09:31.554968 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c060f4-520a-4772-b65c-23b6ea31caf2 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:09:31.555730 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c060f4-520a-4772-b65c-23b6ea31caf2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:09:31.555904 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c060f4-520a-4772-b65c-23b6ea31caf2 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:09:31.558247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:09:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:09:31.565030 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa711da-788f-43fc-843a-d297199bf755 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:09:31.565612 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa711da-788f-43fc-843a-d297199bf755 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:09:31.566008 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa711da-788f-43fc-843a-d297199bf755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:09:31.595701 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-bfa711da-788f-43fc-843a-d297199bf755 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:09:31.597895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-bfa711da-788f-43fc-843a-d297199bf755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:09:31.707685 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:09:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:09:31.910916 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c060f4-520a-4772-b65c-23b6ea31caf2 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:09:31.912742 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-59c060f4-520a-4772-b65c-23b6ea31caf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:09:32.047332 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c060f4-520a-4772-b65c-23b6ea31caf2 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:09:32.057126 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b648cebe-7e1f-4750-abca-3f78937a7d87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:09:32.058368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648cebe-7e1f-4750-abca-3f78937a7d87 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:09:32.058932 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648cebe-7e1f-4750-abca-3f78937a7d87 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:09:32.059445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648cebe-7e1f-4750-abca-3f78937a7d87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:09:32.076447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b648cebe-7e1f-4750-abca-3f78937a7d87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:09:32.080351 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b648cebe-7e1f-4750-abca-3f78937a7d87 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:09:32.098332 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 757/1513] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:09:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:09:33.168811 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f322a-dc88-4a7c-9408-de8624ab44bd None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:09:33.169427 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f322a-dc88-4a7c-9408-de8624ab44bd None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:09:33.169887 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f322a-dc88-4a7c-9408-de8624ab44bd None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:09:33.172335 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:09:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:09:33.179053 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a5514-f12f-4d80-95d3-874823eca47a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:09:33.179889 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a5514-f12f-4d80-95d3-874823eca47a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:09:33.179889 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a5514-f12f-4d80-95d3-874823eca47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:09:33.205274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-675a5514-f12f-4d80-95d3-874823eca47a None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:09:33.206580 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-675a5514-f12f-4d80-95d3-874823eca47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:09:33.217804 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:09:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:09:33.378945 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f322a-dc88-4a7c-9408-de8624ab44bd None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:09:33.396920 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f322a-dc88-4a7c-9408-de8624ab44bd None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:09:33.402768 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb0147-a11e-45fd-8644-732f1c0a80eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:09:33.403458 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb0147-a11e-45fd-8644-732f1c0a80eb service nova] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:09:33.403794 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb0147-a11e-45fd-8644-732f1c0a80eb service nova] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:09:33.404059 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb0147-a11e-45fd-8644-732f1c0a80eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:09:33.414481 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdb0147-a11e-45fd-8644-732f1c0a80eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:09:33.416460 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdb0147-a11e-45fd-8644-732f1c0a80eb service nova] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:09:33.427942 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 758/1516] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:09:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:09:33.500240 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93977b-4b0a-4436-80f1-c39ac7af779a None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:09:33.500547 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93977b-4b0a-4436-80f1-c39ac7af779a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:09:33.500925 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93977b-4b0a-4436-80f1-c39ac7af779a None None] PATH_INFO: `/v3` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:09:33.505656 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 759/1517] 10.222.0.25 () {54 vars in 925 bytes} [Thu Jun 16 16:09:33 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:09:33.517885 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ee364-52bc-49cc-8145-c4c8e8f8fe16 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:09:33.518208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ee364-52bc-49cc-8145-c4c8e8f8fe16 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:09:33.518491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ee364-52bc-49cc-8145-c4c8e8f8fe16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:09:33.540528 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-2e1ee364-52bc-49cc-8145-c4c8e8f8fe16 None None] MFA Rules not processed for user `36b63614297346e29ff276eab444fe5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:09:33.541780 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-2e1ee364-52bc-49cc-8145-c4c8e8f8fe16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:09:33.615157 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 759/1518] 10.222.0.25 () {58 vars in 1015 bytes} [Thu Jun 16 16:09:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:09:37.634590 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b180b82-cf92-4334-929e-af225e7e9491 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:09:37.635261 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b180b82-cf92-4334-929e-af225e7e9491 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:09:37.635261 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b180b82-cf92-4334-929e-af225e7e9491 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:09:37.636602 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 760/1519] 10.222.0.17 () {54 vars in 960 bytes} [Thu Jun 16 16:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:09:37.643978 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6251895-0af4-4af9-a1cc-a41ace96cbe0 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:09:37.644503 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6251895-0af4-4af9-a1cc-a41ace96cbe0 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:09:37.644606 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6251895-0af4-4af9-a1cc-a41ace96cbe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:09:37.671143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-f6251895-0af4-4af9-a1cc-a41ace96cbe0 None None] MFA Rules not processed for user `36b63614297346e29ff276eab444fe5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:09:37.672746 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-f6251895-0af4-4af9-a1cc-a41ace96cbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:09:37.684107 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 760/1520] 10.222.0.17 () {58 vars in 1057 bytes} [Thu Jun 16 16:09:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:09:37.694123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b180b82-cf92-4334-929e-af225e7e9491 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:09:37.711836 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b180b82-cf92-4334-929e-af225e7e9491 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:09:37.716864 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d5f3f-8f23-4d30-b4bc-d19df2fd4df9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36b63614297346e29ff276eab444fe5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:09:37.717518 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d5f3f-8f23-4d30-b4bc-d19df2fd4df9 service placement] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:09:37.717800 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d5f3f-8f23-4d30-b4bc-d19df2fd4df9 service placement] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:09:37.718113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d5f3f-8f23-4d30-b4bc-d19df2fd4df9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:09:37.729393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74d5f3f-8f23-4d30-b4bc-d19df2fd4df9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:09:37.731125 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74d5f3f-8f23-4d30-b4bc-d19df2fd4df9 service placement] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:09:37.744182 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 761/1521] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:09:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:09:44.762508 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbacb2e-959a-439c-870e-535a4350238d None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:09:44.763097 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbacb2e-959a-439c-870e-535a4350238d None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:09:44.763097 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbacb2e-959a-439c-870e-535a4350238d None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:09:44.764481 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:09:44.770066 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc606a-6993-4684-8ed1-e7ed19e827ee None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:09:44.770511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc606a-6993-4684-8ed1-e7ed19e827ee None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:09:44.770632 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc606a-6993-4684-8ed1-e7ed19e827ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:09:44.789728 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-a4cc606a-6993-4684-8ed1-e7ed19e827ee None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:09:44.790999 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a4cc606a-6993-4684-8ed1-e7ed19e827ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:09:44.799012 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:09:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:09:44.956929 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbacb2e-959a-439c-870e-535a4350238d None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:09:44.972562 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbacb2e-959a-439c-870e-535a4350238d None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:09:44.978241 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9637b32a-6d8b-4ba2-9bab-951f9e9e6d13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:09:44.978893 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637b32a-6d8b-4ba2-9bab-951f9e9e6d13 service nova] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:09:44.979174 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637b32a-6d8b-4ba2-9bab-951f9e9e6d13 service nova] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:09:44.979448 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637b32a-6d8b-4ba2-9bab-951f9e9e6d13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:09:44.988891 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9637b32a-6d8b-4ba2-9bab-951f9e9e6d13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:09:44.990617 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9637b32a-6d8b-4ba2-9bab-951f9e9e6d13 service nova] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:09:45.000976 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 762/1524] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:09:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:09:45.934109 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3caf55f-89a3-4970-a74d-8c28f7ad9e10 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:09:45.934688 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3caf55f-89a3-4970-a74d-8c28f7ad9e10 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:09:45.934688 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3caf55f-89a3-4970-a74d-8c28f7ad9e10 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:09:45.936068 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:09:45.941968 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f34fe-91a3-4543-b85f-cd0f98dcd9da None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:09:45.942458 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f34fe-91a3-4543-b85f-cd0f98dcd9da None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:09:45.942559 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f34fe-91a3-4543-b85f-cd0f98dcd9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:09:45.960822 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-009f34fe-91a3-4543-b85f-cd0f98dcd9da None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:09:45.962092 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-009f34fe-91a3-4543-b85f-cd0f98dcd9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:09:45.970628 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:09:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:09:46.130057 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3caf55f-89a3-4970-a74d-8c28f7ad9e10 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:09:46.145744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3caf55f-89a3-4970-a74d-8c28f7ad9e10 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:09:46.151104 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430ab15-886f-4252-be22-f4302588daa8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:09:46.151745 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430ab15-886f-4252-be22-f4302588daa8 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:09:46.152028 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430ab15-886f-4252-be22-f4302588daa8 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:09:46.152299 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430ab15-886f-4252-be22-f4302588daa8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:09:46.161757 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430ab15-886f-4252-be22-f4302588daa8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:09:46.163361 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430ab15-886f-4252-be22-f4302588daa8 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:09:46.174915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 764/1527] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:09:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:09:50.500126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c245f-6598-4497-a15f-29636dc63624 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:09:50.500731 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c245f-6598-4497-a15f-29636dc63624 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:09:50.500731 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c245f-6598-4497-a15f-29636dc63624 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:09:50.502326 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:09:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:09:50.507798 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc632ab8-4f73-46dc-a4cf-bb00ba74dd97 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:09:50.508294 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc632ab8-4f73-46dc-a4cf-bb00ba74dd97 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:09:50.508393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc632ab8-4f73-46dc-a4cf-bb00ba74dd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:09:50.527175 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-fc632ab8-4f73-46dc-a4cf-bb00ba74dd97 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:09:50.528428 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-fc632ab8-4f73-46dc-a4cf-bb00ba74dd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:09:50.539647 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:09:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:09:50.578181 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c245f-6598-4497-a15f-29636dc63624 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:09:50.579952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-867c245f-6598-4497-a15f-29636dc63624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:09:50.640185 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c245f-6598-4497-a15f-29636dc63624 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:09:50.643914 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97952e53-03ec-4d3b-869f-147ec7893c3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:09:50.644567 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97952e53-03ec-4d3b-869f-147ec7893c3c service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:09:50.644853 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97952e53-03ec-4d3b-869f-147ec7893c3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:09:50.645130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97952e53-03ec-4d3b-869f-147ec7893c3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:09:50.654715 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97952e53-03ec-4d3b-869f-147ec7893c3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:09:50.656978 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97952e53-03ec-4d3b-869f-147ec7893c3c service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:09:50.663007 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:09:50.668476 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 765/1530] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:09:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:09:52.890274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fff7a-4c1b-4918-8fb4-6b4b613956e1 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:09:52.891098 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fff7a-4c1b-4918-8fb4-6b4b613956e1 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:09:52.891354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fff7a-4c1b-4918-8fb4-6b4b613956e1 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:09:52.892867 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:09:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:09:52.898283 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde45fd-58a4-4913-8f86-36b23b733569 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:09:52.898728 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde45fd-58a4-4913-8f86-36b23b733569 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:09:52.898864 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde45fd-58a4-4913-8f86-36b23b733569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:09:52.920342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-2cde45fd-58a4-4913-8f86-36b23b733569 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:09:52.922126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-2cde45fd-58a4-4913-8f86-36b23b733569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:09:52.930372 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:09:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:09:52.966894 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6fff7a-4c1b-4918-8fb4-6b4b613956e1 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:09:52.977904 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6fff7a-4c1b-4918-8fb4-6b4b613956e1 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:09:52.981459 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3363a6b8-a1cb-4fa3-8554-636b015380eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:09:52.982124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3363a6b8-a1cb-4fa3-8554-636b015380eb service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:09:52.982412 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3363a6b8-a1cb-4fa3-8554-636b015380eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:09:52.982694 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3363a6b8-a1cb-4fa3-8554-636b015380eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:09:52.992446 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3363a6b8-a1cb-4fa3-8554-636b015380eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:09:52.993897 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3363a6b8-a1cb-4fa3-8554-636b015380eb service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:09:53.004231 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 767/1533] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:09:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:09:57.148824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b111c56-77ba-4754-b11d-8171dc2b2d2a None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:09:57.149410 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b111c56-77ba-4754-b11d-8171dc2b2d2a None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:09:57.149410 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b111c56-77ba-4754-b11d-8171dc2b2d2a None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:09:57.150810 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:09:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:09:57.157539 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370e7b1-2d44-4c0b-b482-e471bd56c750 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:09:57.158062 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370e7b1-2d44-4c0b-b482-e471bd56c750 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:09:57.158183 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370e7b1-2d44-4c0b-b482-e471bd56c750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:09:57.177781 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8370e7b1-2d44-4c0b-b482-e471bd56c750 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:09:57.178808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8370e7b1-2d44-4c0b-b482-e471bd56c750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:09:57.189006 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:09:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:09:57.458188 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b111c56-77ba-4754-b11d-8171dc2b2d2a None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:09:57.459878 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-9b111c56-77ba-4754-b11d-8171dc2b2d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:09:57.520406 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b111c56-77ba-4754-b11d-8171dc2b2d2a None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:09:57.523948 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82345bc-6daa-41ca-97d2-1cca86c14c15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:09:57.524589 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82345bc-6daa-41ca-97d2-1cca86c14c15 service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:09:57.524868 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82345bc-6daa-41ca-97d2-1cca86c14c15 service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:09:57.525141 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82345bc-6daa-41ca-97d2-1cca86c14c15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:09:57.534398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82345bc-6daa-41ca-97d2-1cca86c14c15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:09:57.535844 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82345bc-6daa-41ca-97d2-1cca86c14c15 service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:09:57.545206 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 768/1536] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:09:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:09:59.795329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8e3e4-452a-43d5-9196-cf72986e9e39 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:09:59.797048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8e3e4-452a-43d5-9196-cf72986e9e39 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:09:59.797048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8e3e4-452a-43d5-9196-cf72986e9e39 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:09:59.798063 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:09:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:09:59.804342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62082e0-d803-4a24-83df-4f23f0289953 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:09:59.804843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62082e0-d803-4a24-83df-4f23f0289953 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:09:59.804945 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62082e0-d803-4a24-83df-4f23f0289953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:09:59.824325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-a62082e0-d803-4a24-83df-4f23f0289953 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:09:59.825572 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-a62082e0-d803-4a24-83df-4f23f0289953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:09:59.834337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:09:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:10:00.106493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8e3e4-452a-43d5-9196-cf72986e9e39 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:10:00.108922 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2da8e3e4-452a-43d5-9196-cf72986e9e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:10:00.117590 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=65 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:10:00.124564 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8e3e4-452a-43d5-9196-cf72986e9e39 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:10:00.128468 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c0be8-e1f1-4dd0-ac80-e1708e8bc374 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:10:00.129129 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c0be8-e1f1-4dd0-ac80-e1708e8bc374 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:00.129410 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c0be8-e1f1-4dd0-ac80-e1708e8bc374 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:00.129691 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c0be8-e1f1-4dd0-ac80-e1708e8bc374 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:00.140331 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c0be8-e1f1-4dd0-ac80-e1708e8bc374 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:10:00.141801 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c0be8-e1f1-4dd0-ac80-e1708e8bc374 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:10:00.152306 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 770/1539] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:10:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:10:09.818968 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=32 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:10:10.147742 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:32.757619 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9339f-57ea-44c6-8a13-b6c40e651dc1 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:32.758806 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9339f-57ea-44c6-8a13-b6c40e651dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:32.758948 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9339f-57ea-44c6-8a13-b6c40e651dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:32.790385 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-26a9339f-57ea-44c6-8a13-b6c40e651dc1 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:32.791831 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-26a9339f-57ea-44c6-8a13-b6c40e651dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:32.805018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 770/1540] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:32.816170 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e37d02-2d67-4368-a9db-34bf87ab283d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:32.816805 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e37d02-2d67-4368-a9db-34bf87ab283d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:32.816805 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e37d02-2d67-4368-a9db-34bf87ab283d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:32.843170 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82a4fc-4cce-49f6-8e8b-261a66d1274e None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:32.843494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82a4fc-4cce-49f6-8e8b-261a66d1274e None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:32.843818 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82a4fc-4cce-49f6-8e8b-261a66d1274e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:32.853968 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b8e37d02-2d67-4368-a9db-34bf87ab283d None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:32.855255 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b8e37d02-2d67-4368-a9db-34bf87ab283d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:32.879049 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-6c82a4fc-4cce-49f6-8e8b-261a66d1274e None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:32.879809 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 771/1541] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:32.881018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-6c82a4fc-4cce-49f6-8e8b-261a66d1274e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:32.884104 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e37d02-2d67-4368-a9db-34bf87ab283d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:32.900514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e37d02-2d67-4368-a9db-34bf87ab283d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:32.904212 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 771/1542] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:11:32.913614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427f455-e6e3-49c0-818a-1bf2b114d355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:32.914346 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427f455-e6e3-49c0-818a-1bf2b114d355 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:32.914626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427f455-e6e3-49c0-818a-1bf2b114d355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:32.914903 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427f455-e6e3-49c0-818a-1bf2b114d355 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:32.915833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3427f455-e6e3-49c0-818a-1bf2b114d355 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:11:32.916140 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3427f455-e6e3-49c0-818a-1bf2b114d355 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:32.916935 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be049d56-7c69-44e1-9cbd-a1e916bb6420 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:32.917163 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: WARNING py.warnings [None req-3427f455-e6e3-49c0-818a-1bf2b114d355 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:11:32.917163 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: warnings.warn(msg) Jun 16 16:11:32.917163 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: Jun 16 16:11:32.917412 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be049d56-7c69-44e1-9cbd-a1e916bb6420 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:32.917502 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be049d56-7c69-44e1-9cbd-a1e916bb6420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:32.918109 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3427f455-e6e3-49c0-818a-1bf2b114d355 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:32.932708 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 772/1543] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:11:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:11:32.942151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427f455-e6e3-49c0-818a-1bf2b114d355 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:32.967969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427f455-e6e3-49c0-818a-1bf2b114d355 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:32.969812 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-be049d56-7c69-44e1-9cbd-a1e916bb6420 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:32.971294 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-be049d56-7c69-44e1-9cbd-a1e916bb6420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:32.982569 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbff5b-ab27-496d-92f9-d90b6dde86af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:32.983297 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbff5b-ab27-496d-92f9-d90b6dde86af admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:32.983581 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbff5b-ab27-496d-92f9-d90b6dde86af admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:32.983858 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbff5b-ab27-496d-92f9-d90b6dde86af admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:32.984574 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bbff5b-ab27-496d-92f9-d90b6dde86af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:32.987488 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bbff5b-ab27-496d-92f9-d90b6dde86af admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:32.993061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 772/1544] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:11:32.995597 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7b4ac-8a49-45b5-bad6-107a43638a81 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:32.995894 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7b4ac-8a49-45b5-bad6-107a43638a81 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:32.996177 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7b4ac-8a49-45b5-bad6-107a43638a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:33.028126 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 773/1545] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:11:32 2022] POST /identity/v3/projects => generated 403 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:11:33.029542 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d007c-5932-43a7-b9ec-3fa681624b3a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:33.029833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d007c-5932-43a7-b9ec-3fa681624b3a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:33.030741 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d007c-5932-43a7-b9ec-3fa681624b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:33.037404 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-14c7b4ac-8a49-45b5-bad6-107a43638a81 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:33.038681 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-14c7b4ac-8a49-45b5-bad6-107a43638a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:33.062076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4b4d007c-5932-43a7-b9ec-3fa681624b3a None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:33.063560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4b4d007c-5932-43a7-b9ec-3fa681624b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:33.173882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 773/1546] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:33.173882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7b4ac-8a49-45b5-bad6-107a43638a81 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:33.180586 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 774/1547] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:33.182024 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d007c-5932-43a7-b9ec-3fa681624b3a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:33.184859 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7b4ac-8a49-45b5-bad6-107a43638a81 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:33.192259 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313db0f3-51ae-440a-81ca-d22f91c26138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:33.193382 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313db0f3-51ae-440a-81ca-d22f91c26138 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:33.193899 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313db0f3-51ae-440a-81ca-d22f91c26138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:33.194514 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313db0f3-51ae-440a-81ca-d22f91c26138 admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:33.195669 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313db0f3-51ae-440a-81ca-d22f91c26138 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:11:33.196246 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313db0f3-51ae-440a-81ca-d22f91c26138 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:33.199969 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313db0f3-51ae-440a-81ca-d22f91c26138 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:33.202796 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d007c-5932-43a7-b9ec-3fa681624b3a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:33.207731 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109d576-1745-46c9-a81f-b6e478c6e802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:33.208980 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109d576-1745-46c9-a81f-b6e478c6e802 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:33.209289 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109d576-1745-46c9-a81f-b6e478c6e802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:33.210042 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109d576-1745-46c9-a81f-b6e478c6e802 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:33.210348 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8109d576-1745-46c9-a81f-b6e478c6e802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:33.211374 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: WARNING py.warnings [None req-8109d576-1745-46c9-a81f-b6e478c6e802 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:11:33.211374 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: warnings.warn(msg) Jun 16 16:11:33.211374 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: Jun 16 16:11:33.212615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8109d576-1745-46c9-a81f-b6e478c6e802 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:33.216634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 774/1548] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:11:33.218989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1da553-9aff-4795-9771-204ca02312eb None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:33.219533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1da553-9aff-4795-9771-204ca02312eb None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:33.220039 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1da553-9aff-4795-9771-204ca02312eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:33.242874 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 775/1549] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:11:33 2022] POST /identity/v3/users => generated 302 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:11:33.244556 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58242630-1d3b-4c27-9545-afae2c24944b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:33.244906 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58242630-1d3b-4c27-9545-afae2c24944b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:33.245211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58242630-1d3b-4c27-9545-afae2c24944b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:33.253602 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-ee1da553-9aff-4795-9771-204ca02312eb None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:33.253602 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-ee1da553-9aff-4795-9771-204ca02312eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:33.261638 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 775/1550] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:33.263273 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168154d9-9f76-430b-a7cc-880ab683be40 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:33.263749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168154d9-9f76-430b-a7cc-880ab683be40 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:33.264347 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168154d9-9f76-430b-a7cc-880ab683be40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:33.274454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-58242630-1d3b-4c27-9545-afae2c24944b None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:33.275916 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-58242630-1d3b-4c27-9545-afae2c24944b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:33.285162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-168154d9-9f76-430b-a7cc-880ab683be40 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:33.285985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 776/1551] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:33.287126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-168154d9-9f76-430b-a7cc-880ab683be40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:33.287400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc370c-2632-4fea-ac4a-f77fab243a32 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:33.287765 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc370c-2632-4fea-ac4a-f77fab243a32 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:33.288370 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc370c-2632-4fea-ac4a-f77fab243a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:33.297961 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 776/1552] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:33.298496 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168154d9-9f76-430b-a7cc-880ab683be40 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:33.312504 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168154d9-9f76-430b-a7cc-880ab683be40 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:33.317353 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd6190-0551-44de-81a9-52690d5a14c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:33.318079 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd6190-0551-44de-81a9-52690d5a14c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:33.318391 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd6190-0551-44de-81a9-52690d5a14c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:33.318740 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd6190-0551-44de-81a9-52690d5a14c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:33.319483 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebd6190-0551-44de-81a9-52690d5a14c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:33.319830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-a4dc370c-2632-4fea-ac4a-f77fab243a32 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:33.320558 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: WARNING py.warnings [None req-cebd6190-0551-44de-81a9-52690d5a14c0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:11:33.320558 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: warnings.warn(msg) Jun 16 16:11:33.320558 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: Jun 16 16:11:33.321706 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a4dc370c-2632-4fea-ac4a-f77fab243a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:33.321840 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebd6190-0551-44de-81a9-52690d5a14c0 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:33.332155 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 777/1553] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:33.333656 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc370c-2632-4fea-ac4a-f77fab243a32 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:33.348576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc370c-2632-4fea-ac4a-f77fab243a32 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:33.354417 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7a08f-cdf0-4974-9287-e7da21f882dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:33.355191 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7a08f-cdf0-4974-9287-e7da21f882dd admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:33.355534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7a08f-cdf0-4974-9287-e7da21f882dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:33.355829 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7a08f-cdf0-4974-9287-e7da21f882dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:33.356564 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb7a08f-cdf0-4974-9287-e7da21f882dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:33.358700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb7a08f-cdf0-4974-9287-e7da21f882dd admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:33.367036 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 777/1554] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:11:33 2022] POST /identity/v3/projects => generated 377 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:11:33.367036 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f47ca3-d0ae-4f12-85e7-1ab43f09815a None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:33.367036 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f47ca3-d0ae-4f12-85e7-1ab43f09815a None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:33.367036 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f47ca3-d0ae-4f12-85e7-1ab43f09815a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:33.372413 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 778/1555] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:33.373672 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac6367-a7e0-4172-b8a2-239c77075be2 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:33.374463 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac6367-a7e0-4172-b8a2-239c77075be2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:33.375408 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac6367-a7e0-4172-b8a2-239c77075be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:33.386787 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-f2f47ca3-d0ae-4f12-85e7-1ab43f09815a None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:33.388017 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-f2f47ca3-d0ae-4f12-85e7-1ab43f09815a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:33.394929 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b5ac6367-a7e0-4172-b8a2-239c77075be2 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:33.396124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b5ac6367-a7e0-4172-b8a2-239c77075be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:33.463714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 779/1556] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:33.465303 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a3c20-2751-4b5a-a0b9-c97f22937e61 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:33.465748 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a3c20-2751-4b5a-a0b9-c97f22937e61 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:33.466058 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a3c20-2751-4b5a-a0b9-c97f22937e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:33.477979 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 778/1557] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:33.477979 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f47ca3-d0ae-4f12-85e7-1ab43f09815a None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:33.489632 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f47ca3-d0ae-4f12-85e7-1ab43f09815a None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:33.493023 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ad3a3c20-2751-4b5a-a0b9-c97f22937e61 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:33.494564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd1e29-624a-4753-9ad2-57e1d3592c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:33.494778 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ad3a3c20-2751-4b5a-a0b9-c97f22937e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:33.495250 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd1e29-624a-4753-9ad2-57e1d3592c5e admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:33.495540 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd1e29-624a-4753-9ad2-57e1d3592c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:33.495806 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd1e29-624a-4753-9ad2-57e1d3592c5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:33.496710 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd1e29-624a-4753-9ad2-57e1d3592c5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:11:33.497016 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd1e29-624a-4753-9ad2-57e1d3592c5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:33.498839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd1e29-624a-4753-9ad2-57e1d3592c5e admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:33.504501 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 780/1558] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:33.505454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a3c20-2751-4b5a-a0b9-c97f22937e61 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:33.513781 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 779/1559] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:11:33.515249 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98926d83-6487-46b3-ba09-31e681189cdb None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:33.515708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98926d83-6487-46b3-ba09-31e681189cdb None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:33.516039 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98926d83-6487-46b3-ba09-31e681189cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:33.519279 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a3c20-2751-4b5a-a0b9-c97f22937e61 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:33.527603 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c826af-fbb2-4cbc-8482-3bd59a520761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:33.528576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c826af-fbb2-4cbc-8482-3bd59a520761 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:33.529113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c826af-fbb2-4cbc-8482-3bd59a520761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:33.529620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c826af-fbb2-4cbc-8482-3bd59a520761 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:33.532183 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c826af-fbb2-4cbc-8482-3bd59a520761 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:11:33.532557 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c826af-fbb2-4cbc-8482-3bd59a520761 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:33.534665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c826af-fbb2-4cbc-8482-3bd59a520761 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:33.540665 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-98926d83-6487-46b3-ba09-31e681189cdb None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:33.542291 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-98926d83-6487-46b3-ba09-31e681189cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:33.543463 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:33.554407 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 780/1560] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:33.555372 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98926d83-6487-46b3-ba09-31e681189cdb None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:33.563894 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 781/1561] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:11:33.563894 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c826af-fbb2-4cbc-8482-3bd59a520761 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:33.573620 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98926d83-6487-46b3-ba09-31e681189cdb None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:33.580198 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c826af-fbb2-4cbc-8482-3bd59a520761 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:33.581513 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6051b7-3b5c-4527-9fda-db6e777dff0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:33.582880 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6051b7-3b5c-4527-9fda-db6e777dff0f admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:33.583362 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6051b7-3b5c-4527-9fda-db6e777dff0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:33.583821 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6051b7-3b5c-4527-9fda-db6e777dff0f admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:33.585340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6051b7-3b5c-4527-9fda-db6e777dff0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:33.586384 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17472c-f4c3-437b-9678-11e8c77a5c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:33.587251 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17472c-f4c3-437b-9678-11e8c77a5c0f admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:33.587540 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17472c-f4c3-437b-9678-11e8c77a5c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:33.587813 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17472c-f4c3-437b-9678-11e8c77a5c0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:33.588551 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17472c-f4c3-437b-9678-11e8c77a5c0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:33.589265 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6051b7-3b5c-4527-9fda-db6e777dff0f admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:33.590597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17472c-f4c3-437b-9678-11e8c77a5c0f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:33.604380 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:33.604804 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 782/1562] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:33.605836 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17472c-f4c3-437b-9678-11e8c77a5c0f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:33.619141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17472c-f4c3-437b-9678-11e8c77a5c0f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:33.621480 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 781/1563] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:11:33 2022] POST /identity/v3/users => generated 289 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:11:33.622832 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f315248-bc97-4028-a08f-823c4ce8ad6e None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:33.623121 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f315248-bc97-4028-a08f-823c4ce8ad6e None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:33.623397 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f315248-bc97-4028-a08f-823c4ce8ad6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:33.628946 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ade0a7-7c7d-4960-92de-96b4cff7d3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:33.629979 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ade0a7-7c7d-4960-92de-96b4cff7d3b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:33.630341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ade0a7-7c7d-4960-92de-96b4cff7d3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:33.630653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ade0a7-7c7d-4960-92de-96b4cff7d3b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:33.631641 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ade0a7-7c7d-4960-92de-96b4cff7d3b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:11:33.633365 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ade0a7-7c7d-4960-92de-96b4cff7d3b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:33.634523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ade0a7-7c7d-4960-92de-96b4cff7d3b3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:33.647494 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 783/1564] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:11:33.647951 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-4f315248-bc97-4028-a08f-823c4ce8ad6e None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:33.648365 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ade0a7-7c7d-4960-92de-96b4cff7d3b3 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:33.649611 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-4f315248-bc97-4028-a08f-823c4ce8ad6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:33.659104 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 782/1565] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:33.659955 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f315248-bc97-4028-a08f-823c4ce8ad6e None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:33.664496 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ade0a7-7c7d-4960-92de-96b4cff7d3b3 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:33.669956 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c1884-d0e6-42c2-a489-4eade1d4e667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:33.671791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f315248-bc97-4028-a08f-823c4ce8ad6e None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:33.671924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c1884-d0e6-42c2-a489-4eade1d4e667 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:33.672254 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c1884-d0e6-42c2-a489-4eade1d4e667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:33.672546 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c1884-d0e6-42c2-a489-4eade1d4e667 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:33.673324 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c1884-d0e6-42c2-a489-4eade1d4e667 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:11:33.673630 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c1884-d0e6-42c2-a489-4eade1d4e667 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:33.675349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c1884-d0e6-42c2-a489-4eade1d4e667 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:33.676371 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c402a-c653-4376-b6f0-a7d2ce9cbdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:33.677044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c402a-c653-4376-b6f0-a7d2ce9cbdc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:33.677327 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c402a-c653-4376-b6f0-a7d2ce9cbdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:33.677596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c402a-c653-4376-b6f0-a7d2ce9cbdc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:33.678396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2c402a-c653-4376-b6f0-a7d2ce9cbdc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:33.680307 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2c402a-c653-4376-b6f0-a7d2ce9cbdc3 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:33.688234 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 784/1566] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:11:33.693013 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c1884-d0e6-42c2-a489-4eade1d4e667 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:33.709659 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c1884-d0e6-42c2-a489-4eade1d4e667 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:33.711460 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 783/1567] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:11:33 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:11:33.712965 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ddafc-3988-42a0-9b31-87443091d253 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:33.713316 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ddafc-3988-42a0-9b31-87443091d253 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:33.713665 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ddafc-3988-42a0-9b31-87443091d253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:33.736016 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-e38ddafc-3988-42a0-9b31-87443091d253 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:33.737257 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-e38ddafc-3988-42a0-9b31-87443091d253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:33.791280 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec1a2d-3e86-4d50-9c59-62ea754f0db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:33.792007 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec1a2d-3e86-4d50-9c59-62ea754f0db1 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:33.792349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec1a2d-3e86-4d50-9c59-62ea754f0db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:33.792637 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec1a2d-3e86-4d50-9c59-62ea754f0db1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:33.793338 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eec1a2d-3e86-4d50-9c59-62ea754f0db1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:33.795313 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eec1a2d-3e86-4d50-9c59-62ea754f0db1 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:33.809884 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 784/1568] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:33.810862 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ddafc-3988-42a0-9b31-87443091d253 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:33.822390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ddafc-3988-42a0-9b31-87443091d253 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:33.824279 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 785/1569] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:11:33 2022] POST /identity/v3/projects => generated 375 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:11:33.825204 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec1a2d-3e86-4d50-9c59-62ea754f0db1 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:33.889871 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c56bc83-6c2e-4895-9c4d-7d92521b0a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:33.890954 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56bc83-6c2e-4895-9c4d-7d92521b0a8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:33.891197 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56bc83-6c2e-4895-9c4d-7d92521b0a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:33.891469 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56bc83-6c2e-4895-9c4d-7d92521b0a8a admin admin] PATH_INFO: `/v3/projects/1060bd888a3042f3a9d4b5f299a41ae8/users/6bb6e714e2994475b2eb9f4ee80dc56d/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:33.903887 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec1a2d-3e86-4d50-9c59-62ea754f0db1 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:33.911408 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c56bc83-6c2e-4895-9c4d-7d92521b0a8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1060bd888a3042f3a9d4b5f299a41ae8, user_id=6bb6e714e2994475b2eb9f4ee80dc56d, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:33.911779 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd9ffd-c9b1-41a8-9b23-51b73b1a7bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:33.912509 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd9ffd-c9b1-41a8-9b23-51b73b1a7bb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:33.912686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: WARNING py.warnings [None req-8c56bc83-6c2e-4895-9c4d-7d92521b0a8a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:11:33.912686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: warnings.warn(msg) Jun 16 16:11:33.912686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: Jun 16 16:11:33.912989 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd9ffd-c9b1-41a8-9b23-51b73b1a7bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:33.913149 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd9ffd-c9b1-41a8-9b23-51b73b1a7bb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:33.913840 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd9ffd-c9b1-41a8-9b23-51b73b1a7bb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:33.914981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c56bc83-6c2e-4895-9c4d-7d92521b0a8a admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:33.915514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd9ffd-c9b1-41a8-9b23-51b73b1a7bb6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:33.928177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 786/1570] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:33.929039 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd9ffd-c9b1-41a8-9b23-51b73b1a7bb6 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:33.934222 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 785/1571] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:11:33 2022] PUT /identity/v3/projects/1060bd888a3042f3a9d4b5f299a41ae8/users/6bb6e714e2994475b2eb9f4ee80dc56d/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:11:33.935515 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c56bc83-6c2e-4895-9c4d-7d92521b0a8a admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:33.947820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd9ffd-c9b1-41a8-9b23-51b73b1a7bb6 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:33.997179 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:34.013668 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:34.022250 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cdcd1-e296-416c-9f58-a01fdd63458a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:34.022929 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cdcd1-e296-416c-9f58-a01fdd63458a admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:34.023213 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cdcd1-e296-416c-9f58-a01fdd63458a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:34.023489 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cdcd1-e296-416c-9f58-a01fdd63458a admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:34.024184 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330cdcd1-e296-416c-9f58-a01fdd63458a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:34.026315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330cdcd1-e296-416c-9f58-a01fdd63458a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:34.031056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c56bc83-6c2e-4895-9c4d-7d92521b0a8a admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:34.035956 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423332d-8312-429c-b63d-533b14cc4726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:34.036774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423332d-8312-429c-b63d-533b14cc4726 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:34.037070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423332d-8312-429c-b63d-533b14cc4726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:34.037354 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423332d-8312-429c-b63d-533b14cc4726 admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:34.038145 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423332d-8312-429c-b63d-533b14cc4726 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:11:34.038467 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423332d-8312-429c-b63d-533b14cc4726 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:34.040158 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423332d-8312-429c-b63d-533b14cc4726 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:34.054533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 786/1572] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:11:34.055392 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423332d-8312-429c-b63d-533b14cc4726 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:34.062082 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 787/1573] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:11:33 2022] POST /identity/v3/projects => generated 381 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:11:34.062082 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cdcd1-e296-416c-9f58-a01fdd63458a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:34.074443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423332d-8312-429c-b63d-533b14cc4726 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:34.165366 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4168bc-f998-42cf-8cfb-f899008576f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:34.166094 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4168bc-f998-42cf-8cfb-f899008576f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:34.166430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4168bc-f998-42cf-8cfb-f899008576f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:34.166712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4168bc-f998-42cf-8cfb-f899008576f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:34.167674 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4168bc-f998-42cf-8cfb-f899008576f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:34.171046 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4168bc-f998-42cf-8cfb-f899008576f4 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:34.173144 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cdcd1-e296-416c-9f58-a01fdd63458a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:34.180177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbe142-313f-4cc4-bd3d-da90975b090a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:34.181617 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbe142-313f-4cc4-bd3d-da90975b090a admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:34.182257 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbe142-313f-4cc4-bd3d-da90975b090a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:34.182756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbe142-313f-4cc4-bd3d-da90975b090a admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:34.183686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcbe142-313f-4cc4-bd3d-da90975b090a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:34.185688 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcbe142-313f-4cc4-bd3d-da90975b090a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:34.207265 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 787/1574] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:11:34 2022] POST /identity/v3/projects => generated 405 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:11:34.208327 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4168bc-f998-42cf-8cfb-f899008576f4 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:34.216397 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 788/1575] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:11:34 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:11:34.219852 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbe142-313f-4cc4-bd3d-da90975b090a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:34.229404 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbe142-313f-4cc4-bd3d-da90975b090a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:34.306789 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4168bc-f998-42cf-8cfb-f899008576f4 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:34.314036 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9bb63a-223e-489b-b800-4ec368ec8e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:34.315100 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9bb63a-223e-489b-b800-4ec368ec8e24 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:34.315558 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9bb63a-223e-489b-b800-4ec368ec8e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:34.315952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9bb63a-223e-489b-b800-4ec368ec8e24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:34.316994 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9bb63a-223e-489b-b800-4ec368ec8e24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:11:34.317558 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9bb63a-223e-489b-b800-4ec368ec8e24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:34.318393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd11cf-c4f6-401a-b2ad-221acbaca322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:34.319590 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd11cf-c4f6-401a-b2ad-221acbaca322 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:34.319984 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9bb63a-223e-489b-b800-4ec368ec8e24 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:34.320209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd11cf-c4f6-401a-b2ad-221acbaca322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:34.320740 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd11cf-c4f6-401a-b2ad-221acbaca322 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:34.322234 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd11cf-c4f6-401a-b2ad-221acbaca322 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:34.326761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd11cf-c4f6-401a-b2ad-221acbaca322 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:34.346765 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 788/1576] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:11:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:11:34.346765 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9bb63a-223e-489b-b800-4ec368ec8e24 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:34.363590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9bb63a-223e-489b-b800-4ec368ec8e24 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:34.365879 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 789/1577] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:11:34 2022] POST /identity/v3/users => generated 288 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:11:34.367112 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768dace5-925e-4b32-b59d-84a2b09ef278 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:34.367430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768dace5-925e-4b32-b59d-84a2b09ef278 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:34.367687 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768dace5-925e-4b32-b59d-84a2b09ef278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:34.369565 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f03214-3a0c-4eef-aa1b-74f994475b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:34.370353 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f03214-3a0c-4eef-aa1b-74f994475b80 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:34.370609 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f03214-3a0c-4eef-aa1b-74f994475b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:34.370887 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f03214-3a0c-4eef-aa1b-74f994475b80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:34.371909 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f03214-3a0c-4eef-aa1b-74f994475b80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:34.374192 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f03214-3a0c-4eef-aa1b-74f994475b80 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:34.386855 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 789/1578] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:34.387827 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f03214-3a0c-4eef-aa1b-74f994475b80 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:34.400296 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f03214-3a0c-4eef-aa1b-74f994475b80 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:34.404180 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-768dace5-925e-4b32-b59d-84a2b09ef278 None None] MFA Rules not processed for user `6bb6e714e2994475b2eb9f4ee80dc56d`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:34.406979 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-768dace5-925e-4b32-b59d-84a2b09ef278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:34.408753 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c64ee-51df-451c-a28a-02508fbf5154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:34.409455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c64ee-51df-451c-a28a-02508fbf5154 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:34.409772 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c64ee-51df-451c-a28a-02508fbf5154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:34.410163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c64ee-51df-451c-a28a-02508fbf5154 admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:34.411086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0c64ee-51df-451c-a28a-02508fbf5154 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:34.413096 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0c64ee-51df-451c-a28a-02508fbf5154 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:34.443171 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 790/1579] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:11:34 2022] POST /identity/v3/users => generated 291 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:11:34.450945 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c64ee-51df-451c-a28a-02508fbf5154 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:34.451895 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:34.462435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c64ee-51df-451c-a28a-02508fbf5154 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:34.468434 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b2d4d-0be5-45b2-9e9e-071209161714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:34.469374 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b2d4d-0be5-45b2-9e9e-071209161714 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:34.469714 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b2d4d-0be5-45b2-9e9e-071209161714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:34.470744 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b2d4d-0be5-45b2-9e9e-071209161714 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:34.471584 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367b2d4d-0be5-45b2-9e9e-071209161714 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:34.473700 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367b2d4d-0be5-45b2-9e9e-071209161714 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:34.501024 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 790/1580] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:34 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:34.502505 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768dace5-925e-4b32-b59d-84a2b09ef278 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:34.511935 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 791/1581] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:11:34 2022] POST /identity/v3/projects => generated 389 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:11:34.514540 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b2d4d-0be5-45b2-9e9e-071209161714 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:34.521250 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768dace5-925e-4b32-b59d-84a2b09ef278 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:34.538363 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:34.608491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170f64f-7be7-41b8-9014-3900f447e0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:34.610638 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170f64f-7be7-41b8-9014-3900f447e0ec admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:34.610638 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170f64f-7be7-41b8-9014-3900f447e0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:34.610638 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170f64f-7be7-41b8-9014-3900f447e0ec admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:34.610987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9170f64f-7be7-41b8-9014-3900f447e0ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:34.613041 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9170f64f-7be7-41b8-9014-3900f447e0ec admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:34.621795 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b2d4d-0be5-45b2-9e9e-071209161714 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:34.629630 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc8a7e9-eaf9-4630-8b52-ba37da93466c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:34.630909 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc8a7e9-eaf9-4630-8b52-ba37da93466c admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:34.631406 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc8a7e9-eaf9-4630-8b52-ba37da93466c admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:34.632054 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc8a7e9-eaf9-4630-8b52-ba37da93466c admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:34.633568 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc8a7e9-eaf9-4630-8b52-ba37da93466c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:34.636142 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc8a7e9-eaf9-4630-8b52-ba37da93466c admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:34.644414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 791/1582] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:11:34 2022] POST /identity/v3/users => generated 303 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:11:34.644414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170f64f-7be7-41b8-9014-3900f447e0ec admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:34.650667 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 792/1583] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:34.651638 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc8a7e9-eaf9-4630-8b52-ba37da93466c admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:34.659311 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170f64f-7be7-41b8-9014-3900f447e0ec admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:34.663894 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc8a7e9-eaf9-4630-8b52-ba37da93466c admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:34.664131 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493bb23-114b-4af7-a055-c287b8f29649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:34.664909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493bb23-114b-4af7-a055-c287b8f29649 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:34.665218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493bb23-114b-4af7-a055-c287b8f29649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:34.665532 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493bb23-114b-4af7-a055-c287b8f29649 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:34.666367 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493bb23-114b-4af7-a055-c287b8f29649 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:34.668380 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493bb23-114b-4af7-a055-c287b8f29649 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:34.669361 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc26a42-87ab-4cbc-975a-1c07e5a3277f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:34.670291 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc26a42-87ab-4cbc-975a-1c07e5a3277f admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:34.670606 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc26a42-87ab-4cbc-975a-1c07e5a3277f admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:34.670890 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc26a42-87ab-4cbc-975a-1c07e5a3277f admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:34.671679 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc26a42-87ab-4cbc-975a-1c07e5a3277f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:34.673273 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc26a42-87ab-4cbc-975a-1c07e5a3277f admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:34.685687 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 793/1584] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:34.686622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc26a42-87ab-4cbc-975a-1c07e5a3277f admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:34.700551 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc26a42-87ab-4cbc-975a-1c07e5a3277f admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:34.705242 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 792/1585] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:11:34 2022] POST /identity/v3/projects => generated 383 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:11:34.706376 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493bb23-114b-4af7-a055-c287b8f29649 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:34.707174 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ef44a-d119-4985-8e30-459df1c5e57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:34.708388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ef44a-d119-4985-8e30-459df1c5e57a admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:34.708686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ef44a-d119-4985-8e30-459df1c5e57a admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:34.708967 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ef44a-d119-4985-8e30-459df1c5e57a admin admin] PATH_INFO: `/v3/projects/34323d7e571b4f1db4d6e0c6cd1159cc/users/80386d7afc994931af05fd49fa8043d6/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:34.733882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393ef44a-d119-4985-8e30-459df1c5e57a admin admin] RBAC: Authorizing `identity:create_grant(project_id=34323d7e571b4f1db4d6e0c6cd1159cc, user_id=80386d7afc994931af05fd49fa8043d6, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:34.736836 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393ef44a-d119-4985-8e30-459df1c5e57a admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:34.754406 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 794/1586] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:11:34 2022] PUT /identity/v3/projects/34323d7e571b4f1db4d6e0c6cd1159cc/users/80386d7afc994931af05fd49fa8043d6/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:11:34.755356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ef44a-d119-4985-8e30-459df1c5e57a admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:34.800783 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493bb23-114b-4af7-a055-c287b8f29649 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:34.845251 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ef44a-d119-4985-8e30-459df1c5e57a admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:34.851564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf96dd-6591-4d23-8b62-83b5a239f912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:34.853884 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf96dd-6591-4d23-8b62-83b5a239f912 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:34.853884 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf96dd-6591-4d23-8b62-83b5a239f912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:34.853884 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf96dd-6591-4d23-8b62-83b5a239f912 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:34.853884 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf96dd-6591-4d23-8b62-83b5a239f912 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:34.855553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf96dd-6591-4d23-8b62-83b5a239f912 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:34.855779 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:34.868705 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6fc349-778c-4ff1-b68a-ee8be02e3e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:34.869363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fc349-778c-4ff1-b68a-ee8be02e3e2c admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:34.869673 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fc349-778c-4ff1-b68a-ee8be02e3e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:34.869983 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fc349-778c-4ff1-b68a-ee8be02e3e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:34.870797 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6fc349-778c-4ff1-b68a-ee8be02e3e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:34.872413 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6fc349-778c-4ff1-b68a-ee8be02e3e2c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:34.885012 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 793/1587] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:34.886184 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6fc349-778c-4ff1-b68a-ee8be02e3e2c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:34.892854 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 795/1588] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:11:34 2022] POST /identity/v3/projects => generated 405 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:11:34.892854 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf96dd-6591-4d23-8b62-83b5a239f912 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:34.898783 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6fc349-778c-4ff1-b68a-ee8be02e3e2c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:34.966802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5d44c-74db-4a87-9d26-fbf2c4d531b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:34.967963 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5d44c-74db-4a87-9d26-fbf2c4d531b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:34.968250 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5d44c-74db-4a87-9d26-fbf2c4d531b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:34.968627 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5d44c-74db-4a87-9d26-fbf2c4d531b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:34.969467 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5d44c-74db-4a87-9d26-fbf2c4d531b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:34.971457 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5d44c-74db-4a87-9d26-fbf2c4d531b9 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:34.976023 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf96dd-6591-4d23-8b62-83b5a239f912 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:34.981332 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58e4e0-5c17-42e7-b0cd-4473b6d24c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:34.982072 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58e4e0-5c17-42e7-b0cd-4473b6d24c39 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:34.982361 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58e4e0-5c17-42e7-b0cd-4473b6d24c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:34.982684 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58e4e0-5c17-42e7-b0cd-4473b6d24c39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:34.984036 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e58e4e0-5c17-42e7-b0cd-4473b6d24c39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:34.986074 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e58e4e0-5c17-42e7-b0cd-4473b6d24c39 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:34.987664 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:34.994011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 794/1589] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:11:34 2022] POST /identity/v3/users => generated 295 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:11:34.995144 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5d44c-74db-4a87-9d26-fbf2c4d531b9 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:35.001437 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 796/1590] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:35.002537 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58e4e0-5c17-42e7-b0cd-4473b6d24c39 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:35.005995 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5d44c-74db-4a87-9d26-fbf2c4d531b9 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:35.010923 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c531f00-1562-43db-bfa2-e239ed30c8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:35.011634 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c531f00-1562-43db-bfa2-e239ed30c8ce admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:35.011937 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c531f00-1562-43db-bfa2-e239ed30c8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:35.012244 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c531f00-1562-43db-bfa2-e239ed30c8ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:35.012793 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58e4e0-5c17-42e7-b0cd-4473b6d24c39 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:35.013000 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c531f00-1562-43db-bfa2-e239ed30c8ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:35.014756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c531f00-1562-43db-bfa2-e239ed30c8ce admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:35.017861 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd47fe-e411-4a94-aa1e-e51db197ad05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:35.018597 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd47fe-e411-4a94-aa1e-e51db197ad05 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:35.018877 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd47fe-e411-4a94-aa1e-e51db197ad05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:35.019155 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd47fe-e411-4a94-aa1e-e51db197ad05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:35.019874 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd47fe-e411-4a94-aa1e-e51db197ad05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:35.021489 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd47fe-e411-4a94-aa1e-e51db197ad05 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:35.027873 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 795/1591] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:35.028632 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c531f00-1562-43db-bfa2-e239ed30c8ce admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:35.036140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 797/1592] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:35.036140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b3512-1274-4fe3-902a-80ba25e065f6 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:35.036140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b3512-1274-4fe3-902a-80ba25e065f6 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:35.036140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b3512-1274-4fe3-902a-80ba25e065f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:35.043254 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c531f00-1562-43db-bfa2-e239ed30c8ce admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:35.049483 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f576055-fd3d-4c27-bb68-ecd274262147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:35.050208 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f576055-fd3d-4c27-bb68-ecd274262147 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:35.050496 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f576055-fd3d-4c27-bb68-ecd274262147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:35.050773 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f576055-fd3d-4c27-bb68-ecd274262147 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:35.051472 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f576055-fd3d-4c27-bb68-ecd274262147 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:35.053431 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f576055-fd3d-4c27-bb68-ecd274262147 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:35.054156 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-0a6b3512-1274-4fe3-902a-80ba25e065f6 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:35.055402 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-0a6b3512-1274-4fe3-902a-80ba25e065f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:35.064173 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 798/1593] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:35.067680 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b3512-1274-4fe3-902a-80ba25e065f6 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:35.075981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b3512-1274-4fe3-902a-80ba25e065f6 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:35.080193 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 796/1594] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:11:35 2022] POST /identity/v3/users => generated 292 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:11:35.081248 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f576055-fd3d-4c27-bb68-ecd274262147 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:35.081654 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd4af0-9f25-4f9d-a34f-3a3a40488b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:35.082444 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd4af0-9f25-4f9d-a34f-3a3a40488b44 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:35.082733 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd4af0-9f25-4f9d-a34f-3a3a40488b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:35.083010 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd4af0-9f25-4f9d-a34f-3a3a40488b44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:35.083749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd4af0-9f25-4f9d-a34f-3a3a40488b44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:35.085376 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd4af0-9f25-4f9d-a34f-3a3a40488b44 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:35.091619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f576055-fd3d-4c27-bb68-ecd274262147 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:35.097351 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c5e93-9b12-41d6-8c3b-7640a7b2f8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:35.097529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 799/1595] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:35.098056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd4af0-9f25-4f9d-a34f-3a3a40488b44 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:35.098185 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c5e93-9b12-41d6-8c3b-7640a7b2f8e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:35.098476 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c5e93-9b12-41d6-8c3b-7640a7b2f8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:35.098807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c5e93-9b12-41d6-8c3b-7640a7b2f8e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:35.099543 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c5e93-9b12-41d6-8c3b-7640a7b2f8e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:35.101571 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c5e93-9b12-41d6-8c3b-7640a7b2f8e7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:35.110362 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd4af0-9f25-4f9d-a34f-3a3a40488b44 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:35.115281 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d901846-1377-46e8-a683-278017371e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:35.115964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d901846-1377-46e8-a683-278017371e1f admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:35.116254 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d901846-1377-46e8-a683-278017371e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:35.116535 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d901846-1377-46e8-a683-278017371e1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:35.117273 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d901846-1377-46e8-a683-278017371e1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:35.119202 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d901846-1377-46e8-a683-278017371e1f admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:35.131404 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 800/1596] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:35.132560 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d901846-1377-46e8-a683-278017371e1f admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:35.137615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 797/1597] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:11:35 2022] POST /identity/v3/users => generated 309 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:11:35.138388 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c5e93-9b12-41d6-8c3b-7640a7b2f8e7 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:35.144481 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d901846-1377-46e8-a683-278017371e1f admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:35.149184 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c5e93-9b12-41d6-8c3b-7640a7b2f8e7 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:35.151018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf331609-5637-4910-9776-105a87b049b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:35.151698 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf331609-5637-4910-9776-105a87b049b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:35.151985 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf331609-5637-4910-9776-105a87b049b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:35.152382 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf331609-5637-4910-9776-105a87b049b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:35.153127 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf331609-5637-4910-9776-105a87b049b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:35.154193 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10981d5-c606-4837-84b6-faaeb92b4d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:35.155314 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf331609-5637-4910-9776-105a87b049b6 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:35.155938 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10981d5-c606-4837-84b6-faaeb92b4d38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:35.156820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10981d5-c606-4837-84b6-faaeb92b4d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:35.156954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10981d5-c606-4837-84b6-faaeb92b4d38 admin admin] PATH_INFO: `/v3/projects/4e4bfae59d38461c839fc9a35eb4d6cf/users/4d48e41e19584c5c8773c9f445355ff2/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:35.166650 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 801/1598] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:35.167665 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf331609-5637-4910-9776-105a87b049b6 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:35.178679 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf331609-5637-4910-9776-105a87b049b6 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:35.180564 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10981d5-c606-4837-84b6-faaeb92b4d38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e4bfae59d38461c839fc9a35eb4d6cf, user_id=4d48e41e19584c5c8773c9f445355ff2, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:35.183450 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10981d5-c606-4837-84b6-faaeb92b4d38 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:35.195231 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43bd8d-0d0d-49cb-985f-05522de38051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:35.196167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43bd8d-0d0d-49cb-985f-05522de38051 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:35.196591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43bd8d-0d0d-49cb-985f-05522de38051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:35.197003 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43bd8d-0d0d-49cb-985f-05522de38051 admin admin] PATH_INFO: `/v3/projects/6c59d2c3976348ac8b7ded542eb6a4cc/users/b4741eaedd5545e59b1dbb7e5ed14d86/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:35.199947 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 798/1599] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:11:35 2022] PUT /identity/v3/projects/4e4bfae59d38461c839fc9a35eb4d6cf/users/4d48e41e19584c5c8773c9f445355ff2/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:11:35.201069 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10981d5-c606-4837-84b6-faaeb92b4d38 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:35.218177 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43bd8d-0d0d-49cb-985f-05522de38051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c59d2c3976348ac8b7ded542eb6a4cc, user_id=b4741eaedd5545e59b1dbb7e5ed14d86, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:35.221362 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43bd8d-0d0d-49cb-985f-05522de38051 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:35.237202 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 802/1600] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:11:35 2022] PUT /identity/v3/projects/6c59d2c3976348ac8b7ded542eb6a4cc/users/b4741eaedd5545e59b1dbb7e5ed14d86/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:11:35.238088 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43bd8d-0d0d-49cb-985f-05522de38051 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:35.292282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10981d5-c606-4837-84b6-faaeb92b4d38 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:35.318159 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43bd8d-0d0d-49cb-985f-05522de38051 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:35.320610 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:35.323362 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb7a8bc-cedc-4e10-b265-5d1f01d115ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:35.324050 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7a8bc-cedc-4e10-b265-5d1f01d115ac admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:35.324339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7a8bc-cedc-4e10-b265-5d1f01d115ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:35.324635 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7a8bc-cedc-4e10-b265-5d1f01d115ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:35.325507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb7a8bc-cedc-4e10-b265-5d1f01d115ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:35.328551 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb7a8bc-cedc-4e10-b265-5d1f01d115ac admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:35.340019 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 803/1601] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:35.341221 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb7a8bc-cedc-4e10-b265-5d1f01d115ac admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:35.352398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb7a8bc-cedc-4e10-b265-5d1f01d115ac admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:35.358363 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2b8a23-edc0-44f1-b131-2ef547ef0f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:35.359118 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b8a23-edc0-44f1-b131-2ef547ef0f80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:35.359401 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b8a23-edc0-44f1-b131-2ef547ef0f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:35.359681 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b8a23-edc0-44f1-b131-2ef547ef0f80 admin admin] PATH_INFO: `/v3/projects/e2442bae81a1434fba74abd03bc94eec/users/cafedd57e92740f7bbfaf5f6df41d71e/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:35.376970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d5684-64a8-4620-9e10-bdaa34ace3fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:35.377873 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d5684-64a8-4620-9e10-bdaa34ace3fe service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:35.378364 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2b8a23-edc0-44f1-b131-2ef547ef0f80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2442bae81a1434fba74abd03bc94eec, user_id=cafedd57e92740f7bbfaf5f6df41d71e, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:35.378534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d5684-64a8-4620-9e10-bdaa34ace3fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:35.378782 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d5684-64a8-4620-9e10-bdaa34ace3fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:35.381286 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2b8a23-edc0-44f1-b131-2ef547ef0f80 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:35.391241 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d5684-64a8-4620-9e10-bdaa34ace3fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:35.393323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d5684-64a8-4620-9e10-bdaa34ace3fe service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:35.397565 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 804/1602] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:11:35 2022] PUT /identity/v3/projects/e2442bae81a1434fba74abd03bc94eec/users/cafedd57e92740f7bbfaf5f6df41d71e/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:11:35.398488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2b8a23-edc0-44f1-b131-2ef547ef0f80 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:35.413620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 799/1603] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:11:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:11:35.415088 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d5684-64a8-4620-9e10-bdaa34ace3fe service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:35.429405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d5684-64a8-4620-9e10-bdaa34ace3fe service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:35.488484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:35.513385 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2b8a23-edc0-44f1-b131-2ef547ef0f80 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:35.522520 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc596e8b-ffc3-4075-ab71-d5308997cbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:35.523728 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc596e8b-ffc3-4075-ab71-d5308997cbc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:35.524274 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc596e8b-ffc3-4075-ab71-d5308997cbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:35.524757 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc596e8b-ffc3-4075-ab71-d5308997cbc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:35.526287 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc596e8b-ffc3-4075-ab71-d5308997cbc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:35.529586 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc596e8b-ffc3-4075-ab71-d5308997cbc1 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:35.531831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48536f3-4506-42af-936b-755acd571304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:35.532879 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48536f3-4506-42af-936b-755acd571304 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:35.533244 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48536f3-4506-42af-936b-755acd571304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:35.533866 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48536f3-4506-42af-936b-755acd571304 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:35.535611 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48536f3-4506-42af-936b-755acd571304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:35.537757 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48536f3-4506-42af-936b-755acd571304 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:35.550603 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 805/1604] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:35.557733 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 800/1605] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:35.557733 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c68e7-72f2-42c0-bb90-0da189a1fe91 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:35.557733 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c68e7-72f2-42c0-bb90-0da189a1fe91 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:35.557733 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c68e7-72f2-42c0-bb90-0da189a1fe91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:35.558451 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc596e8b-ffc3-4075-ab71-d5308997cbc1 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:35.575982 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc596e8b-ffc3-4075-ab71-d5308997cbc1 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:35.583445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-6a1c68e7-72f2-42c0-bb90-0da189a1fe91 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:35.584827 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac7d9e-8dbf-4b35-ad39-f41aa755bcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:35.585794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6a1c68e7-72f2-42c0-bb90-0da189a1fe91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:35.586657 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac7d9e-8dbf-4b35-ad39-f41aa755bcbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:35.587300 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac7d9e-8dbf-4b35-ad39-f41aa755bcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:35.587901 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac7d9e-8dbf-4b35-ad39-f41aa755bcbb admin admin] PATH_INFO: `/v3/projects/cbf2b6097fc344fab753b7a12d32f49e/users/6c8d6307851948b5809b123667be0190/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:35.605168 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 801/1606] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:35.607335 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f639b-bacd-4325-ad9a-a8c7efd889d7 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:35.607614 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac7d9e-8dbf-4b35-ad39-f41aa755bcbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbf2b6097fc344fab753b7a12d32f49e, user_id=6c8d6307851948b5809b123667be0190, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:35.607826 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f639b-bacd-4325-ad9a-a8c7efd889d7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:35.608800 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f639b-bacd-4325-ad9a-a8c7efd889d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:35.612526 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac7d9e-8dbf-4b35-ad39-f41aa755bcbb admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:35.641795 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 806/1607] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:11:35 2022] PUT /identity/v3/projects/cbf2b6097fc344fab753b7a12d32f49e/users/6c8d6307851948b5809b123667be0190/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:11:35.641795 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac7d9e-8dbf-4b35-ad39-f41aa755bcbb admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:35.679461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0b5f639b-bacd-4325-ad9a-a8c7efd889d7 None None] MFA Rules not processed for user `b4741eaedd5545e59b1dbb7e5ed14d86`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:35.681061 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0b5f639b-bacd-4325-ad9a-a8c7efd889d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:35.725863 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac7d9e-8dbf-4b35-ad39-f41aa755bcbb admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:35.735025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0d05a2-a5ac-4475-95d2-81f8ca458581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:35.736121 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d05a2-a5ac-4475-95d2-81f8ca458581 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:35.737013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d05a2-a5ac-4475-95d2-81f8ca458581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:35.737639 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d05a2-a5ac-4475-95d2-81f8ca458581 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:35.739005 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0d05a2-a5ac-4475-95d2-81f8ca458581 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:35.740907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0d05a2-a5ac-4475-95d2-81f8ca458581 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:35.753817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 807/1608] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:35.755305 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b65d28-f5df-4a30-af90-1bf25d97d6ee None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:35.756137 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b65d28-f5df-4a30-af90-1bf25d97d6ee None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:35.756137 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b65d28-f5df-4a30-af90-1bf25d97d6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:35.767420 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 802/1609] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:35.768611 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f639b-bacd-4325-ad9a-a8c7efd889d7 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:35.787738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-b4b65d28-f5df-4a30-af90-1bf25d97d6ee None None] MFA Rules not processed for user `cafedd57e92740f7bbfaf5f6df41d71e`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:35.788956 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-b4b65d28-f5df-4a30-af90-1bf25d97d6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:35.794784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f639b-bacd-4325-ad9a-a8c7efd889d7 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:35.805657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45286e9-8b62-4a54-8e67-910e327bdb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:35.806611 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45286e9-8b62-4a54-8e67-910e327bdb73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:35.806824 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45286e9-8b62-4a54-8e67-910e327bdb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:35.807085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45286e9-8b62-4a54-8e67-910e327bdb73 admin admin] PATH_INFO: `/v3/projects/6eee49063a5847dba6ffb520d0149d88/users/515114f0a842463fb2d1265ee33db8eb/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:35.832095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45286e9-8b62-4a54-8e67-910e327bdb73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eee49063a5847dba6ffb520d0149d88, user_id=515114f0a842463fb2d1265ee33db8eb, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:35.836197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45286e9-8b62-4a54-8e67-910e327bdb73 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:35.855290 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 803/1610] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:11:35 2022] PUT /identity/v3/projects/6eee49063a5847dba6ffb520d0149d88/users/515114f0a842463fb2d1265ee33db8eb/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:11:35.856783 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45286e9-8b62-4a54-8e67-910e327bdb73 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:35.882466 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 808/1611] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:35.883635 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b65d28-f5df-4a30-af90-1bf25d97d6ee None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:35.885243 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-b4b65d28-f5df-4a30-af90-1bf25d97d6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:35.893993 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:35.966836 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45286e9-8b62-4a54-8e67-910e327bdb73 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:35.972470 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b65d28-f5df-4a30-af90-1bf25d97d6ee None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:35.977634 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27287a9-2972-48b4-b30f-63f849ac476d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:35.979227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:35.981240 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27287a9-2972-48b4-b30f-63f849ac476d admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:35.982098 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27287a9-2972-48b4-b30f-63f849ac476d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:35.982547 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27287a9-2972-48b4-b30f-63f849ac476d admin admin] PATH_INFO: `/v3/projects/fb9a0fb7ff804caba3b1a6c4cdf7779a/users/108497109866423aaada9b1a367780a4/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:36.008272 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27287a9-2972-48b4-b30f-63f849ac476d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb9a0fb7ff804caba3b1a6c4cdf7779a, user_id=108497109866423aaada9b1a367780a4, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:36.011229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27287a9-2972-48b4-b30f-63f849ac476d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:36.029550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 804/1612] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:11:35 2022] PUT /identity/v3/projects/fb9a0fb7ff804caba3b1a6c4cdf7779a/users/108497109866423aaada9b1a367780a4/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:11:36.032066 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c995b-22d7-43db-a0d4-4959f48cba5c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:36.032471 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c995b-22d7-43db-a0d4-4959f48cba5c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:36.032827 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c995b-22d7-43db-a0d4-4959f48cba5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:36.045304 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d5e1d-ed30-468c-83c1-fbe92bf43836 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:36.047079 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d5e1d-ed30-468c-83c1-fbe92bf43836 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:36.047550 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d5e1d-ed30-468c-83c1-fbe92bf43836 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:36.047838 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d5e1d-ed30-468c-83c1-fbe92bf43836 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:36.054526 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-262c995b-22d7-43db-a0d4-4959f48cba5c None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:36.055867 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-262c995b-22d7-43db-a0d4-4959f48cba5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:36.059400 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d5e1d-ed30-468c-83c1-fbe92bf43836 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:36.060899 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d5e1d-ed30-468c-83c1-fbe92bf43836 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:36.072758 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 809/1613] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:11:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:11:36.073911 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d5e1d-ed30-468c-83c1-fbe92bf43836 service neutron] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:36.086483 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d5e1d-ed30-468c-83c1-fbe92bf43836 service neutron] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:36.143602 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 805/1614] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:36.145770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505e1c5-c777-4b58-bb5d-4f36880012fe None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:36.150489 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505e1c5-c777-4b58-bb5d-4f36880012fe None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:36.152309 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505e1c5-c777-4b58-bb5d-4f36880012fe None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:36.155461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 806/1615] 10.222.0.17 () {54 vars in 961 bytes} [Thu Jun 16 16:11:36 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:11:36.159906 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106152b-cacb-4f29-90fd-98bc8c9108f0 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:36.160414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106152b-cacb-4f29-90fd-98bc8c9108f0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:36.160878 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106152b-cacb-4f29-90fd-98bc8c9108f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:36.180806 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632970e-a40e-4990-b84c-915fe3dfe104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:36.181805 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632970e-a40e-4990-b84c-915fe3dfe104 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:36.182160 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632970e-a40e-4990-b84c-915fe3dfe104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:36.182443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632970e-a40e-4990-b84c-915fe3dfe104 admin admin] PATH_INFO: `/v3/projects/b091584c67814e4d8f7e88f0596880e3/users/2c34f05634d34da29088bcf75069958d/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:36.192044 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9106152b-cacb-4f29-90fd-98bc8c9108f0 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:36.193523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9106152b-cacb-4f29-90fd-98bc8c9108f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:36.206341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 807/1616] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:36.207331 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106152b-cacb-4f29-90fd-98bc8c9108f0 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:36.215561 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4632970e-a40e-4990-b84c-915fe3dfe104 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b091584c67814e4d8f7e88f0596880e3, user_id=2c34f05634d34da29088bcf75069958d, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:36.220710 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106152b-cacb-4f29-90fd-98bc8c9108f0 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:36.221802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4632970e-a40e-4990-b84c-915fe3dfe104 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:36.226937 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11a24b-34da-4733-a5eb-3e6e1a06b295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:36.228291 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11a24b-34da-4733-a5eb-3e6e1a06b295 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:36.228907 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11a24b-34da-4733-a5eb-3e6e1a06b295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:36.229421 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11a24b-34da-4733-a5eb-3e6e1a06b295 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:36.230306 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d11a24b-34da-4733-a5eb-3e6e1a06b295 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:36.233311 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d11a24b-34da-4733-a5eb-3e6e1a06b295 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:36.242498 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 810/1617] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:11:36 2022] PUT /identity/v3/projects/b091584c67814e4d8f7e88f0596880e3/users/2c34f05634d34da29088bcf75069958d/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:11:36.243703 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632970e-a40e-4990-b84c-915fe3dfe104 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:36.279590 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 808/1618] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:11:36 2022] POST /identity/v3/projects => generated 381 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:11:36.282694 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5dcba7-697e-4bed-8bfe-98bee4e42ee0 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:36.283757 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5dcba7-697e-4bed-8bfe-98bee4e42ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:36.284360 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5dcba7-697e-4bed-8bfe-98bee4e42ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:36.318444 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-3d5dcba7-697e-4bed-8bfe-98bee4e42ee0 None None] MFA Rules not processed for user `c3645b1a0be34c01ac87c734fcd85ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:36.320120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3d5dcba7-697e-4bed-8bfe-98bee4e42ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:36.375360 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632970e-a40e-4990-b84c-915fe3dfe104 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:36.453898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 809/1619] 10.222.0.17 () {58 vars in 1058 bytes} [Thu Jun 16 16:11:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:36.455207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5dcba7-697e-4bed-8bfe-98bee4e42ee0 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:36.456224 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:36.469526 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:36.500675 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3365cd-fb99-4270-9c5d-8fc0eb902c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:36.501376 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3365cd-fb99-4270-9c5d-8fc0eb902c55 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:36.501841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3365cd-fb99-4270-9c5d-8fc0eb902c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:36.502821 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3365cd-fb99-4270-9c5d-8fc0eb902c55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:36.503609 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3365cd-fb99-4270-9c5d-8fc0eb902c55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:36.505913 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3365cd-fb99-4270-9c5d-8fc0eb902c55 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:36.528622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 811/1620] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:36.529659 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3365cd-fb99-4270-9c5d-8fc0eb902c55 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:36.602914 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3365cd-fb99-4270-9c5d-8fc0eb902c55 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:36.619578 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5dcba7-697e-4bed-8bfe-98bee4e42ee0 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:36.624583 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8427ed-a220-48f4-beba-9681b5708afd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:36.625310 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8427ed-a220-48f4-beba-9681b5708afd service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:36.625598 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8427ed-a220-48f4-beba-9681b5708afd service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:36.625883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8427ed-a220-48f4-beba-9681b5708afd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:36.626142 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f649d92-0881-4079-892d-c07ddc913c98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:36.626719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f649d92-0881-4079-892d-c07ddc913c98 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:36.627063 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f649d92-0881-4079-892d-c07ddc913c98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:36.627648 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f649d92-0881-4079-892d-c07ddc913c98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:36.647410 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8427ed-a220-48f4-beba-9681b5708afd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:36.648935 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8427ed-a220-48f4-beba-9681b5708afd service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:36.651804 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f649d92-0881-4079-892d-c07ddc913c98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:36.653336 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f649d92-0881-4079-892d-c07ddc913c98 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:36.662826 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 812/1621] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:11:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:11:36.664543 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679cc37-29a6-4545-9167-83c3bb8d96c6 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:36.665159 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679cc37-29a6-4545-9167-83c3bb8d96c6 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:36.665604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679cc37-29a6-4545-9167-83c3bb8d96c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:36.670813 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 810/1622] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:11:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:11:36.672655 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f649d92-0881-4079-892d-c07ddc913c98 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:36.687035 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f649d92-0881-4079-892d-c07ddc913c98 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:36.690486 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-e679cc37-29a6-4545-9167-83c3bb8d96c6 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:36.691888 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-e679cc37-29a6-4545-9167-83c3bb8d96c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:36.693297 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df63e08-b2d4-433f-9034-28f80e6ab7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:36.694218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df63e08-b2d4-433f-9034-28f80e6ab7a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:36.694583 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df63e08-b2d4-433f-9034-28f80e6ab7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:36.694913 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df63e08-b2d4-433f-9034-28f80e6ab7a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:36.695743 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df63e08-b2d4-433f-9034-28f80e6ab7a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:36.697810 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df63e08-b2d4-433f-9034-28f80e6ab7a6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:36.700609 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 813/1623] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:36.701483 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679cc37-29a6-4545-9167-83c3bb8d96c6 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:36.724002 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679cc37-29a6-4545-9167-83c3bb8d96c6 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:36.731462 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 811/1624] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:11:36 2022] POST /identity/v3/users => generated 297 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:11:36.731462 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df63e08-b2d4-433f-9034-28f80e6ab7a6 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:36.731741 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e81e65-e37c-4c10-870d-3e92d2f5404b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:36.731741 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e81e65-e37c-4c10-870d-3e92d2f5404b service nova] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:36.732375 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e81e65-e37c-4c10-870d-3e92d2f5404b service nova] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:36.732841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e81e65-e37c-4c10-870d-3e92d2f5404b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:36.743647 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df63e08-b2d4-433f-9034-28f80e6ab7a6 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:36.752165 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd8a2c-c52a-4472-a209-896b2c66a519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:36.753681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd8a2c-c52a-4472-a209-896b2c66a519 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:36.754118 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd8a2c-c52a-4472-a209-896b2c66a519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:36.754548 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd8a2c-c52a-4472-a209-896b2c66a519 admin admin] PATH_INFO: `/v3/projects/fb9a0fb7ff804caba3b1a6c4cdf7779a/users/108497109866423aaada9b1a367780a4/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:36.758166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd8a2c-c52a-4472-a209-896b2c66a519 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb9a0fb7ff804caba3b1a6c4cdf7779a, user_id=108497109866423aaada9b1a367780a4, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:36.762609 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd8a2c-c52a-4472-a209-896b2c66a519 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:36.782684 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 812/1625] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:11:36 2022] PUT /identity/v3/projects/fb9a0fb7ff804caba3b1a6c4cdf7779a/users/108497109866423aaada9b1a367780a4/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:11:36.784413 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd8a2c-c52a-4472-a209-896b2c66a519 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:36.839381 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e81e65-e37c-4c10-870d-3e92d2f5404b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:36.841117 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e81e65-e37c-4c10-870d-3e92d2f5404b service nova] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:36.854915 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 814/1626] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:11:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:11:36.855897 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e81e65-e37c-4c10-870d-3e92d2f5404b service nova] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:36.866030 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e81e65-e37c-4c10-870d-3e92d2f5404b service nova] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:36.971960 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd8a2c-c52a-4472-a209-896b2c66a519 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:36.985985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c919f919-0281-449b-8ab4-ddf437e3e51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:36.986706 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919f919-0281-449b-8ab4-ddf437e3e51d admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:36.986998 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919f919-0281-449b-8ab4-ddf437e3e51d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:36.987282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919f919-0281-449b-8ab4-ddf437e3e51d admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:36.991590 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c919f919-0281-449b-8ab4-ddf437e3e51d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:36.995376 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c919f919-0281-449b-8ab4-ddf437e3e51d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:37.002388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dbda1-7287-4c5c-b0dd-7cc388ff5c67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:37.004360 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dbda1-7287-4c5c-b0dd-7cc388ff5c67 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:37.004659 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dbda1-7287-4c5c-b0dd-7cc388ff5c67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:37.004945 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dbda1-7287-4c5c-b0dd-7cc388ff5c67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:37.034987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 813/1627] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:37.038290 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759216b-faac-4133-973f-a9da74c2713e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:37.038626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759216b-faac-4133-973f-a9da74c2713e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:37.040423 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849dbda1-7287-4c5c-b0dd-7cc388ff5c67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:37.041698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759216b-faac-4133-973f-a9da74c2713e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:37.044695 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849dbda1-7287-4c5c-b0dd-7cc388ff5c67 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:37.071387 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 815/1628] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:11:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:11:37.072079 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dbda1-7287-4c5c-b0dd-7cc388ff5c67 service neutron] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:37.096774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dbda1-7287-4c5c-b0dd-7cc388ff5c67 service neutron] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:37.106770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0759216b-faac-4133-973f-a9da74c2713e None None] MFA Rules not processed for user `108497109866423aaada9b1a367780a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:37.107816 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0759216b-faac-4133-973f-a9da74c2713e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:37.112487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68dfde-8f8a-43dc-8192-edadd926ce64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:37.113403 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68dfde-8f8a-43dc-8192-edadd926ce64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:37.113714 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68dfde-8f8a-43dc-8192-edadd926ce64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:37.114061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68dfde-8f8a-43dc-8192-edadd926ce64 admin admin] PATH_INFO: `/v3/projects/eca1f139068b4b248291159017100c54/users/6578c95cfb4349e5be5d9eb6fee7c26a/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:37.156462 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:37.172470 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68dfde-8f8a-43dc-8192-edadd926ce64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eca1f139068b4b248291159017100c54, user_id=6578c95cfb4349e5be5d9eb6fee7c26a, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:37.175171 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68dfde-8f8a-43dc-8192-edadd926ce64 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:37.206518 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 816/1629] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:11:37 2022] PUT /identity/v3/projects/eca1f139068b4b248291159017100c54/users/6578c95cfb4349e5be5d9eb6fee7c26a/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:11:37.212435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68dfde-8f8a-43dc-8192-edadd926ce64 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:37.307617 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:37.328909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 814/1630] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:37 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:37.343835 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0759216b-faac-4133-973f-a9da74c2713e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:37.412824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68dfde-8f8a-43dc-8192-edadd926ce64 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:37.425009 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468c7fc-3827-44a9-a94c-7503b91c07c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:37.425806 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468c7fc-3827-44a9-a94c-7503b91c07c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:37.426203 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468c7fc-3827-44a9-a94c-7503b91c07c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:37.426555 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468c7fc-3827-44a9-a94c-7503b91c07c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:37.427630 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7468c7fc-3827-44a9-a94c-7503b91c07c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:37.429401 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7468c7fc-3827-44a9-a94c-7503b91c07c9 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:37.457378 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 817/1631] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:37.462565 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468c7fc-3827-44a9-a94c-7503b91c07c9 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:37.492003 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0759216b-faac-4133-973f-a9da74c2713e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:37.493469 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468c7fc-3827-44a9-a94c-7503b91c07c9 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:37.500299 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89489d30-9f71-405b-98f5-ae9c20514747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:37.501952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89489d30-9f71-405b-98f5-ae9c20514747 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:37.502647 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89489d30-9f71-405b-98f5-ae9c20514747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:37.503194 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89489d30-9f71-405b-98f5-ae9c20514747 admin admin] PATH_INFO: `/v3/projects/eca1f139068b4b248291159017100c54/users/6578c95cfb4349e5be5d9eb6fee7c26a/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:37.508890 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89489d30-9f71-405b-98f5-ae9c20514747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eca1f139068b4b248291159017100c54, user_id=6578c95cfb4349e5be5d9eb6fee7c26a, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:37.518322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89489d30-9f71-405b-98f5-ae9c20514747 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:37.552317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 818/1632] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:11:37 2022] PUT /identity/v3/projects/eca1f139068b4b248291159017100c54/users/6578c95cfb4349e5be5d9eb6fee7c26a/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:11:37.564632 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3a4cb-7391-4900-910c-9cd801181693 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:37.568124 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3a4cb-7391-4900-910c-9cd801181693 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:37.568610 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3a4cb-7391-4900-910c-9cd801181693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:37.673325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-2ad3a4cb-7391-4900-910c-9cd801181693 None None] MFA Rules not processed for user `6578c95cfb4349e5be5d9eb6fee7c26a`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:37.679163 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315ba1b-f88e-48e3-b376-5ffe3e0615c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:37.681699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315ba1b-f88e-48e3-b376-5ffe3e0615c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:37.682377 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-2ad3a4cb-7391-4900-910c-9cd801181693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:37.684462 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315ba1b-f88e-48e3-b376-5ffe3e0615c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:37.685122 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315ba1b-f88e-48e3-b376-5ffe3e0615c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:37.853693 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315ba1b-f88e-48e3-b376-5ffe3e0615c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:37.855333 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315ba1b-f88e-48e3-b376-5ffe3e0615c9 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:37.877677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 815/1633] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:11:37 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:11:37.901529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 819/1634] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:37 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:11:37.908375 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c34c34-37ac-4347-bc88-e772af394850 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:37.908375 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c34c34-37ac-4347-bc88-e772af394850 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:37.908724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c34c34-37ac-4347-bc88-e772af394850 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:37.913814 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 816/1635] 10.222.0.17 () {54 vars in 961 bytes} [Thu Jun 16 16:11:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:11:37.921617 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3a4cb-7391-4900-910c-9cd801181693 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:37.963619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb35719-4c2c-40d2-9d9a-1614a7ad3bc5 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:37.964188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb35719-4c2c-40d2-9d9a-1614a7ad3bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:37.965146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb35719-4c2c-40d2-9d9a-1614a7ad3bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:38.027807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-dfb35719-4c2c-40d2-9d9a-1614a7ad3bc5 None None] MFA Rules not processed for user `c3645b1a0be34c01ac87c734fcd85ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:38.032501 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-dfb35719-4c2c-40d2-9d9a-1614a7ad3bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:38.120244 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3a4cb-7391-4900-910c-9cd801181693 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:38.128808 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685ff10-a429-44c3-bab9-7421bd9ea444 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:38.130508 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685ff10-a429-44c3-bab9-7421bd9ea444 service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:38.130801 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685ff10-a429-44c3-bab9-7421bd9ea444 service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:38.131335 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685ff10-a429-44c3-bab9-7421bd9ea444 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:38.145711 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685ff10-a429-44c3-bab9-7421bd9ea444 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:38.148428 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685ff10-a429-44c3-bab9-7421bd9ea444 service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:38.164668 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:38.188812 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 820/1636] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:11:37 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:11:38.239080 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:38.261071 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685ff10-a429-44c3-bab9-7421bd9ea444 service cinder] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:38.300745 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 817/1637] 10.222.0.17 () {58 vars in 1058 bytes} [Thu Jun 16 16:11:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 16 16:11:38.313787 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229808a-2048-48cc-9693-23f215f37f2b None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:38.314456 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229808a-2048-48cc-9693-23f215f37f2b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:38.314905 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229808a-2048-48cc-9693-23f215f37f2b None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:38.316920 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 818/1638] 10.222.0.17 () {54 vars in 912 bytes} [Thu Jun 16 16:11:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:11:38.322276 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43eac3-9837-458f-8a09-289fcfb65b23 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:38.323583 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43eac3-9837-458f-8a09-289fcfb65b23 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:38.324023 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43eac3-9837-458f-8a09-289fcfb65b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:38.398239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0e43eac3-9837-458f-8a09-289fcfb65b23 None None] MFA Rules not processed for user `c3645b1a0be34c01ac87c734fcd85ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:38.403192 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0e43eac3-9837-458f-8a09-289fcfb65b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:38.441139 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 819/1639] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 16:11:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:11:38.442985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43eac3-9837-458f-8a09-289fcfb65b23 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:38.463317 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43eac3-9837-458f-8a09-289fcfb65b23 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:38.476077 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb2a70-3749-4c08-8c9c-8de0f4976873 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:38.477017 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb2a70-3749-4c08-8c9c-8de0f4976873 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:38.477824 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb2a70-3749-4c08-8c9c-8de0f4976873 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:38.478465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb2a70-3749-4c08-8c9c-8de0f4976873 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:38.486280 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685ff10-a429-44c3-bab9-7421bd9ea444 service cinder] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:38.496175 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412be0c-2e18-4709-90a3-2e06f8170945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:38.500156 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412be0c-2e18-4709-90a3-2e06f8170945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:38.500870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412be0c-2e18-4709-90a3-2e06f8170945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:38.500870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412be0c-2e18-4709-90a3-2e06f8170945 admin admin] PATH_INFO: `/v3/users/6bb6e714e2994475b2eb9f4ee80dc56d` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:38.511089 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e412be0c-2e18-4709-90a3-2e06f8170945 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bb6e714e2994475b2eb9f4ee80dc56d)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:38.514169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: WARNING py.warnings [None req-e412be0c-2e18-4709-90a3-2e06f8170945 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:11:38.514169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: warnings.warn(msg) Jun 16 16:11:38.514169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: Jun 16 16:11:38.517493 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e412be0c-2e18-4709-90a3-2e06f8170945 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:38.654104 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-e412be0c-2e18-4709-90a3-2e06f8170945 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bb6e714e2994475b2eb9f4ee80dc56d'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:38.657904 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb2a70-3749-4c08-8c9c-8de0f4976873 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:38.659787 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb2a70-3749-4c08-8c9c-8de0f4976873 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:38.668320 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-e412be0c-2e18-4709-90a3-2e06f8170945 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bb6e714e2994475b2eb9f4ee80dc56d'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:38.681483 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 820/1640] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:11:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:11:38.683217 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb2a70-3749-4c08-8c9c-8de0f4976873 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:38.699356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-e412be0c-2e18-4709-90a3-2e06f8170945 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bb6e714e2994475b2eb9f4ee80dc56d'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:38.700498 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-e412be0c-2e18-4709-90a3-2e06f8170945 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bb6e714e2994475b2eb9f4ee80dc56d'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:38.706338 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb2a70-3749-4c08-8c9c-8de0f4976873 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:38.714962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-e412be0c-2e18-4709-90a3-2e06f8170945 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bb6e714e2994475b2eb9f4ee80dc56d'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:38.716418 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbf87a-7522-401d-85d9-90240b3e8162 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:38.717321 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbf87a-7522-401d-85d9-90240b3e8162 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:38.718627 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbf87a-7522-401d-85d9-90240b3e8162 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:38.718986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbf87a-7522-401d-85d9-90240b3e8162 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:38.724314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f6bbf87a-7522-401d-85d9-90240b3e8162 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:38.728090 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 821/1641] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:11:38 2022] DELETE /identity/v3/users/6bb6e714e2994475b2eb9f4ee80dc56d => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:11:38.729791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e8c6b-3657-46ed-89da-53083e173be0 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:38.734907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e8c6b-3657-46ed-89da-53083e173be0 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:38.735302 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e8c6b-3657-46ed-89da-53083e173be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:38.818828 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-f94e8c6b-3657-46ed-89da-53083e173be0 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:38.818828 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-f94e8c6b-3657-46ed-89da-53083e173be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:38.820054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bbf87a-7522-401d-85d9-90240b3e8162 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:38.820054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bbf87a-7522-401d-85d9-90240b3e8162 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:38.820054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 821/1642] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:11:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:11:38.912702 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 822/1643] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:38.977184 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbf87a-7522-401d-85d9-90240b3e8162 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:38.979128 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f6bbf87a-7522-401d-85d9-90240b3e8162 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:39.157311 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f71a0-d15d-4b31-8562-57234db396b2 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:39.158038 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f71a0-d15d-4b31-8562-57234db396b2 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:39.158038 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f71a0-d15d-4b31-8562-57234db396b2 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:39.160007 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 823/1644] 10.222.0.17 () {54 vars in 912 bytes} [Thu Jun 16 16:11:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:11:39.168154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbf87a-7522-401d-85d9-90240b3e8162 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:39.172536 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabe99e-f956-4249-be7f-9e25fc2594a3 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:39.172934 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabe99e-f956-4249-be7f-9e25fc2594a3 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:39.173222 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabe99e-f956-4249-be7f-9e25fc2594a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:39.216548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-4cabe99e-f956-4249-be7f-9e25fc2594a3 None None] MFA Rules not processed for user `c3645b1a0be34c01ac87c734fcd85ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:39.222679 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-4cabe99e-f956-4249-be7f-9e25fc2594a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:39.263126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:39.322564 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb52230-58d3-4ed9-8d2e-bc53d6528e78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:39.323461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb52230-58d3-4ed9-8d2e-bc53d6528e78 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:39.324176 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb52230-58d3-4ed9-8d2e-bc53d6528e78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:39.324600 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb52230-58d3-4ed9-8d2e-bc53d6528e78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:39.339709 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb52230-58d3-4ed9-8d2e-bc53d6528e78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:39.341558 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb52230-58d3-4ed9-8d2e-bc53d6528e78 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:39.364908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 822/1645] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:11:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:11:39.468658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 824/1646] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 16:11:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:11:39.500464 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb52230-58d3-4ed9-8d2e-bc53d6528e78 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:39.504979 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1fb52230-58d3-4ed9-8d2e-bc53d6528e78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:39.590737 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:39.651597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb52230-58d3-4ed9-8d2e-bc53d6528e78 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:39.661317 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e406b-edd4-4212-8470-fbce8df640d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:39.677945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e406b-edd4-4212-8470-fbce8df640d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:39.679391 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e406b-edd4-4212-8470-fbce8df640d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:39.680844 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e406b-edd4-4212-8470-fbce8df640d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:39.708107 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e406b-edd4-4212-8470-fbce8df640d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:39.710192 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e406b-edd4-4212-8470-fbce8df640d6 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:39.742840 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 823/1647] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:11:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:11:39.997096 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cabe99e-f956-4249-be7f-9e25fc2594a3 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:39.999820 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-4cabe99e-f956-4249-be7f-9e25fc2594a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:40.034789 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cabe99e-f956-4249-be7f-9e25fc2594a3 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:40.054211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec12e5-86ce-4c97-b318-66173a5592c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:40.055095 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec12e5-86ce-4c97-b318-66173a5592c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:40.055413 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec12e5-86ce-4c97-b318-66173a5592c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:40.055684 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec12e5-86ce-4c97-b318-66173a5592c8 admin admin] PATH_INFO: `/v3/users/b4741eaedd5545e59b1dbb7e5ed14d86` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:40.060012 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec12e5-86ce-4c97-b318-66173a5592c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4741eaedd5545e59b1dbb7e5ed14d86)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:40.062963 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec12e5-86ce-4c97-b318-66173a5592c8 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:40.193316 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-77ec12e5-86ce-4c97-b318-66173a5592c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4741eaedd5545e59b1dbb7e5ed14d86'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:40.212714 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-77ec12e5-86ce-4c97-b318-66173a5592c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4741eaedd5545e59b1dbb7e5ed14d86'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:40.235451 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-77ec12e5-86ce-4c97-b318-66173a5592c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4741eaedd5545e59b1dbb7e5ed14d86'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:40.236539 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-77ec12e5-86ce-4c97-b318-66173a5592c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4741eaedd5545e59b1dbb7e5ed14d86'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:40.250359 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-77ec12e5-86ce-4c97-b318-66173a5592c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4741eaedd5545e59b1dbb7e5ed14d86'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:40.260465 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 825/1648] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:11:39 2022] DELETE /identity/v3/users/b4741eaedd5545e59b1dbb7e5ed14d86 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:11:40.266087 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67cefc-6d2a-49eb-938d-97887f5732f8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:40.266747 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67cefc-6d2a-49eb-938d-97887f5732f8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:40.266883 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67cefc-6d2a-49eb-938d-97887f5732f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:40.295836 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-bb67cefc-6d2a-49eb-938d-97887f5732f8 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:40.297295 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bb67cefc-6d2a-49eb-938d-97887f5732f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:40.470609 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 824/1649] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:40.503750 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec12e5-86ce-4c97-b318-66173a5592c8 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:40.508984 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-77ec12e5-86ce-4c97-b318-66173a5592c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:40.684470 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec12e5-86ce-4c97-b318-66173a5592c8 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:40.695522 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306bd2c-f6ea-4af0-ad05-13ce28ca7d82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:40.697193 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306bd2c-f6ea-4af0-ad05-13ce28ca7d82 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:40.699079 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306bd2c-f6ea-4af0-ad05-13ce28ca7d82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:40.699460 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306bd2c-f6ea-4af0-ad05-13ce28ca7d82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:40.722868 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306bd2c-f6ea-4af0-ad05-13ce28ca7d82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:40.727431 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306bd2c-f6ea-4af0-ad05-13ce28ca7d82 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:40.756905 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 826/1650] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:11:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:11:41.577064 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67cefc-6d2a-49eb-938d-97887f5732f8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:41.579164 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bb67cefc-6d2a-49eb-938d-97887f5732f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:41.607413 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67cefc-6d2a-49eb-938d-97887f5732f8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:41.622193 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba03ba05-0c3c-4974-9928-1e2bc03e56c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:41.623141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03ba05-0c3c-4974-9928-1e2bc03e56c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:41.623494 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03ba05-0c3c-4974-9928-1e2bc03e56c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:41.623783 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03ba05-0c3c-4974-9928-1e2bc03e56c8 admin admin] PATH_INFO: `/v3/projects/1060bd888a3042f3a9d4b5f299a41ae8` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:41.625466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba03ba05-0c3c-4974-9928-1e2bc03e56c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1060bd888a3042f3a9d4b5f299a41ae8)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:41.626533 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: WARNING py.warnings [None req-ba03ba05-0c3c-4974-9928-1e2bc03e56c8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:11:41.626533 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: warnings.warn(msg) Jun 16 16:11:41.626533 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: Jun 16 16:11:41.628063 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba03ba05-0c3c-4974-9928-1e2bc03e56c8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:41.802185 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ba03ba05-0c3c-4974-9928-1e2bc03e56c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1060bd888a3042f3a9d4b5f299a41ae8', 'request_id': 'req-ba03ba05-0c3c-4974-9928-1e2bc03e56c8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:41.834243 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ba03ba05-0c3c-4974-9928-1e2bc03e56c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1060bd888a3042f3a9d4b5f299a41ae8', 'request_id': 'req-ba03ba05-0c3c-4974-9928-1e2bc03e56c8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:41.856848 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ba03ba05-0c3c-4974-9928-1e2bc03e56c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1060bd888a3042f3a9d4b5f299a41ae8', 'request_id': 'req-ba03ba05-0c3c-4974-9928-1e2bc03e56c8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:41.892963 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ba03ba05-0c3c-4974-9928-1e2bc03e56c8 admin admin] The token cache is being invalidate because project 1060bd888a3042f3a9d4b5f299a41ae8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:11:41.893252 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ba03ba05-0c3c-4974-9928-1e2bc03e56c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:41.894476 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ba03ba05-0c3c-4974-9928-1e2bc03e56c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:41.910890 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 825/1651] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:11:41 2022] DELETE /identity/v3/projects/1060bd888a3042f3a9d4b5f299a41ae8 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:11:41.914231 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306bd2c-f6ea-4af0-ad05-13ce28ca7d82 service neutron] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:41.947567 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306bd2c-f6ea-4af0-ad05-13ce28ca7d82 service neutron] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:41.992512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:42.072421 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97998091-b919-47f8-bdaa-c637e38fbf11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:42.074217 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97998091-b919-47f8-bdaa-c637e38fbf11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:42.074574 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97998091-b919-47f8-bdaa-c637e38fbf11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:42.074920 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97998091-b919-47f8-bdaa-c637e38fbf11 admin admin] PATH_INFO: `/v3/users/108497109866423aaada9b1a367780a4` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:42.076367 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97998091-b919-47f8-bdaa-c637e38fbf11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=108497109866423aaada9b1a367780a4)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:42.080074 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97998091-b919-47f8-bdaa-c637e38fbf11 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:42.105230 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18e11c-a70e-4add-8bc5-53ad50eb669b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:42.106134 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18e11c-a70e-4add-8bc5-53ad50eb669b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:42.106632 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18e11c-a70e-4add-8bc5-53ad50eb669b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:42.153113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-3b18e11c-a70e-4add-8bc5-53ad50eb669b None None] MFA Rules not processed for user `6c8d6307851948b5809b123667be0190`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:42.155818 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3b18e11c-a70e-4add-8bc5-53ad50eb669b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:42.201251 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-97998091-b919-47f8-bdaa-c637e38fbf11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '108497109866423aaada9b1a367780a4'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:42.213755 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-97998091-b919-47f8-bdaa-c637e38fbf11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '108497109866423aaada9b1a367780a4'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:42.249079 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-97998091-b919-47f8-bdaa-c637e38fbf11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '108497109866423aaada9b1a367780a4'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:42.251403 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-97998091-b919-47f8-bdaa-c637e38fbf11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '108497109866423aaada9b1a367780a4'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:42.269181 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-97998091-b919-47f8-bdaa-c637e38fbf11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '108497109866423aaada9b1a367780a4'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:42.287760 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 827/1652] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:11:41 2022] DELETE /identity/v3/users/108497109866423aaada9b1a367780a4 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:11:42.289499 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:42.313206 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 826/1653] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:42 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:11:42.320537 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97998091-b919-47f8-bdaa-c637e38fbf11 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:42.323455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-97998091-b919-47f8-bdaa-c637e38fbf11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:42.430723 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18e11c-a70e-4add-8bc5-53ad50eb669b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:42.432623 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3b18e11c-a70e-4add-8bc5-53ad50eb669b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:42.465813 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18e11c-a70e-4add-8bc5-53ad50eb669b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:42.511380 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97998091-b919-47f8-bdaa-c637e38fbf11 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:42.521301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f01cd-b049-44cb-b809-2f8f84acd81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:42.522558 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f01cd-b049-44cb-b809-2f8f84acd81b admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:42.522856 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f01cd-b049-44cb-b809-2f8f84acd81b admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:42.523141 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f01cd-b049-44cb-b809-2f8f84acd81b admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:42.523892 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f01cd-b049-44cb-b809-2f8f84acd81b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:42.526886 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f01cd-b049-44cb-b809-2f8f84acd81b admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:42.579232 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 828/1654] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:11:42 2022] POST /identity/v3/projects => generated 405 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:11:42.587003 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f01cd-b049-44cb-b809-2f8f84acd81b admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:42.668851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db50201-9e32-4b1a-9313-de71158e11f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:42.672596 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db50201-9e32-4b1a-9313-de71158e11f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:42.674148 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db50201-9e32-4b1a-9313-de71158e11f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:42.675490 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:42.676149 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db50201-9e32-4b1a-9313-de71158e11f9 admin admin] PATH_INFO: `/v3/projects/6c59d2c3976348ac8b7ded542eb6a4cc` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:42.683656 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db50201-9e32-4b1a-9313-de71158e11f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c59d2c3976348ac8b7ded542eb6a4cc)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:42.693001 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db50201-9e32-4b1a-9313-de71158e11f9 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:42.770817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f01cd-b049-44cb-b809-2f8f84acd81b admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:42.796218 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5fb720-6eab-4cf0-856e-0b00c1d42ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:42.797101 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fb720-6eab-4cf0-856e-0b00c1d42ae5 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:42.797648 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fb720-6eab-4cf0-856e-0b00c1d42ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:42.801053 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fb720-6eab-4cf0-856e-0b00c1d42ae5 admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:42.802816 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5fb720-6eab-4cf0-856e-0b00c1d42ae5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:42.809760 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5fb720-6eab-4cf0-856e-0b00c1d42ae5 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:42.875220 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 829/1655] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:11:42 2022] POST /identity/v3/users => generated 309 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:11:42.878869 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5fb720-6eab-4cf0-856e-0b00c1d42ae5 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:42.900956 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8db50201-9e32-4b1a-9313-de71158e11f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c59d2c3976348ac8b7ded542eb6a4cc', 'request_id': 'req-8db50201-9e32-4b1a-9313-de71158e11f9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:42.921532 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5fb720-6eab-4cf0-856e-0b00c1d42ae5 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:42.928753 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8db50201-9e32-4b1a-9313-de71158e11f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c59d2c3976348ac8b7ded542eb6a4cc', 'request_id': 'req-8db50201-9e32-4b1a-9313-de71158e11f9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:42.949581 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8db50201-9e32-4b1a-9313-de71158e11f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c59d2c3976348ac8b7ded542eb6a4cc', 'request_id': 'req-8db50201-9e32-4b1a-9313-de71158e11f9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:43.007085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8db50201-9e32-4b1a-9313-de71158e11f9 admin admin] The token cache is being invalidate because project 6c59d2c3976348ac8b7ded542eb6a4cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:11:43.008414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8db50201-9e32-4b1a-9313-de71158e11f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:43.010914 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8db50201-9e32-4b1a-9313-de71158e11f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:43.027306 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 827/1656] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:11:42 2022] DELETE /identity/v3/projects/6c59d2c3976348ac8b7ded542eb6a4cc => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:11:43.030978 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ffe0af-a64d-4b36-96ce-c380a232d2c4 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:43.035183 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ffe0af-a64d-4b36-96ce-c380a232d2c4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:43.036028 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ffe0af-a64d-4b36-96ce-c380a232d2c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:43.121629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57eafa2-9e38-495c-91f8-2e7cd9d84d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:43.128478 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57eafa2-9e38-495c-91f8-2e7cd9d84d7f admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:43.129764 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57eafa2-9e38-495c-91f8-2e7cd9d84d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:43.129914 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57eafa2-9e38-495c-91f8-2e7cd9d84d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:43.131499 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57eafa2-9e38-495c-91f8-2e7cd9d84d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:43.137608 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57eafa2-9e38-495c-91f8-2e7cd9d84d7f admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:43.180046 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 830/1657] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:43.181578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57eafa2-9e38-495c-91f8-2e7cd9d84d7f admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:43.190954 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-a57eafa2-9e38-495c-91f8-2e7cd9d84d7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:43.247875 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57eafa2-9e38-495c-91f8-2e7cd9d84d7f admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:43.271653 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36dbe9b-29b9-46f3-b6f7-21d7163c633e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:43.272502 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36dbe9b-29b9-46f3-b6f7-21d7163c633e admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:43.272784 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36dbe9b-29b9-46f3-b6f7-21d7163c633e admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:43.273058 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36dbe9b-29b9-46f3-b6f7-21d7163c633e admin admin] PATH_INFO: `/v3/projects/810abdc029ae4b71b375590bbd1a8ceb/users/742ae6608b514ebe83d21a936266848a/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:43.308822 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36dbe9b-29b9-46f3-b6f7-21d7163c633e admin admin] RBAC: Authorizing `identity:create_grant(project_id=810abdc029ae4b71b375590bbd1a8ceb, user_id=742ae6608b514ebe83d21a936266848a, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:43.315427 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36dbe9b-29b9-46f3-b6f7-21d7163c633e admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:43.332619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-62ffe0af-a64d-4b36-96ce-c380a232d2c4 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:43.333870 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-62ffe0af-a64d-4b36-96ce-c380a232d2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:43.344652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 831/1658] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:11:43 2022] PUT /identity/v3/projects/810abdc029ae4b71b375590bbd1a8ceb/users/742ae6608b514ebe83d21a936266848a/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:11:43.346190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36dbe9b-29b9-46f3-b6f7-21d7163c633e admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:43.523272 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36dbe9b-29b9-46f3-b6f7-21d7163c633e admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:43.532979 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d417e3-8181-45f6-a044-290ce40b8cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:43.534008 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d417e3-8181-45f6-a044-290ce40b8cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:43.534374 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d417e3-8181-45f6-a044-290ce40b8cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:43.534724 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d417e3-8181-45f6-a044-290ce40b8cd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:43.536646 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d417e3-8181-45f6-a044-290ce40b8cd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:43.540416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d417e3-8181-45f6-a044-290ce40b8cd9 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:43.543294 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 828/1659] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 515 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:43.552432 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67671995-bf80-4f78-996a-78077ea57d88 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:43.553253 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67671995-bf80-4f78-996a-78077ea57d88 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:43.553786 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67671995-bf80-4f78-996a-78077ea57d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:43.575674 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 832/1660] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:43.579291 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d417e3-8181-45f6-a044-290ce40b8cd9 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:43.603719 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d417e3-8181-45f6-a044-290ce40b8cd9 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:43.622530 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-67671995-bf80-4f78-996a-78077ea57d88 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:43.626942 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-67671995-bf80-4f78-996a-78077ea57d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:43.627415 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90080e-e720-467c-88ff-b180764c8821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:43.631170 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90080e-e720-467c-88ff-b180764c8821 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:43.631812 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90080e-e720-467c-88ff-b180764c8821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:43.632517 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90080e-e720-467c-88ff-b180764c8821 admin admin] PATH_INFO: `/v3/projects/810abdc029ae4b71b375590bbd1a8ceb/users/742ae6608b514ebe83d21a936266848a/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:43.638297 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90080e-e720-467c-88ff-b180764c8821 admin admin] RBAC: Authorizing `identity:create_grant(project_id=810abdc029ae4b71b375590bbd1a8ceb, user_id=742ae6608b514ebe83d21a936266848a, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:43.646575 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90080e-e720-467c-88ff-b180764c8821 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:43.647573 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 829/1661] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:43.652373 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67671995-bf80-4f78-996a-78077ea57d88 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:43.664557 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67671995-bf80-4f78-996a-78077ea57d88 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:43.676089 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 833/1662] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:11:43 2022] PUT /identity/v3/projects/810abdc029ae4b71b375590bbd1a8ceb/users/742ae6608b514ebe83d21a936266848a/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:11:43.703447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:43.873967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff529b94-86b2-4d9c-b83e-5f254b00798d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:43.874924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff529b94-86b2-4d9c-b83e-5f254b00798d admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:43.875214 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff529b94-86b2-4d9c-b83e-5f254b00798d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:43.875546 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff529b94-86b2-4d9c-b83e-5f254b00798d admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:43.876372 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff529b94-86b2-4d9c-b83e-5f254b00798d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:11:43.876603 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff529b94-86b2-4d9c-b83e-5f254b00798d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:43.878493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff529b94-86b2-4d9c-b83e-5f254b00798d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:43.900523 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90080e-e720-467c-88ff-b180764c8821 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:43.900887 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 830/1663] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:11:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:11:43.939363 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90080e-e720-467c-88ff-b180764c8821 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:43.965352 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1134171-7140-49e6-bb2e-ef410a134be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:43.966148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1134171-7140-49e6-bb2e-ef410a134be0 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:43.966438 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1134171-7140-49e6-bb2e-ef410a134be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:43.966720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1134171-7140-49e6-bb2e-ef410a134be0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:43.968862 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1134171-7140-49e6-bb2e-ef410a134be0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:43.971010 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1134171-7140-49e6-bb2e-ef410a134be0 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:44.004733 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:44.037537 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 834/1664] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:11:43 2022] POST /identity/v3/projects => generated 373 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:11:44.043445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff529b94-86b2-4d9c-b83e-5f254b00798d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:44.059017 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff529b94-86b2-4d9c-b83e-5f254b00798d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:44.303249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63f95a-d34f-4015-af37-4995628931aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:44.304475 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63f95a-d34f-4015-af37-4995628931aa admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:44.305123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63f95a-d34f-4015-af37-4995628931aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:44.305493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63f95a-d34f-4015-af37-4995628931aa admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:44.306822 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a63f95a-d34f-4015-af37-4995628931aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:44.310742 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a63f95a-d34f-4015-af37-4995628931aa admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:44.410085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 831/1665] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:11:44 2022] POST /identity/v3/users => generated 287 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:11:44.413908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1134171-7140-49e6-bb2e-ef410a134be0 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:44.429388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1134171-7140-49e6-bb2e-ef410a134be0 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:44.454270 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce80d9e-36b2-4968-917c-77e8ac3240e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:44.454991 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce80d9e-36b2-4968-917c-77e8ac3240e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:44.455355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce80d9e-36b2-4968-917c-77e8ac3240e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:44.455639 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce80d9e-36b2-4968-917c-77e8ac3240e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:44.456563 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce80d9e-36b2-4968-917c-77e8ac3240e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:44.458322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce80d9e-36b2-4968-917c-77e8ac3240e1 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:44.489381 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 835/1666] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:44 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:44.494665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63f95a-d34f-4015-af37-4995628931aa admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:44.528653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63f95a-d34f-4015-af37-4995628931aa admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:44.548031 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f9f90-6883-428d-8de6-285f69ce8266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:44.549146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f9f90-6883-428d-8de6-285f69ce8266 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:44.549667 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f9f90-6883-428d-8de6-285f69ce8266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:44.550288 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f9f90-6883-428d-8de6-285f69ce8266 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:44.551402 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f9f90-6883-428d-8de6-285f69ce8266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:44.553781 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f9f90-6883-428d-8de6-285f69ce8266 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:44.618139 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 832/1667] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:44 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:44.623444 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce80d9e-36b2-4968-917c-77e8ac3240e1 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:44.665225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce80d9e-36b2-4968-917c-77e8ac3240e1 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:44.700280 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc236da5-4f64-4ed7-9120-5bc8372df62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:44.702697 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc236da5-4f64-4ed7-9120-5bc8372df62a admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:44.703094 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc236da5-4f64-4ed7-9120-5bc8372df62a admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:44.703472 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc236da5-4f64-4ed7-9120-5bc8372df62a admin admin] PATH_INFO: `/v3/projects/aefe3b0db19c4d88a4559ea34ba3b7e0/users/9528dc0eb5024b659c437751bd5b306b/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:44.838867 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc236da5-4f64-4ed7-9120-5bc8372df62a admin admin] RBAC: Authorizing `identity:create_grant(project_id=aefe3b0db19c4d88a4559ea34ba3b7e0, user_id=9528dc0eb5024b659c437751bd5b306b, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:44.853283 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc236da5-4f64-4ed7-9120-5bc8372df62a admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:44.946353 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 836/1668] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:11:44 2022] PUT /identity/v3/projects/aefe3b0db19c4d88a4559ea34ba3b7e0/users/9528dc0eb5024b659c437751bd5b306b/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:11:44.953125 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997546de-6a88-425a-a590-f893ac39c5a2 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:44.959705 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997546de-6a88-425a-a590-f893ac39c5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:44.959705 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997546de-6a88-425a-a590-f893ac39c5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:45.091297 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-997546de-6a88-425a-a590-f893ac39c5a2 None None] MFA Rules not processed for user `9528dc0eb5024b659c437751bd5b306b`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:45.095190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-997546de-6a88-425a-a590-f893ac39c5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:45.392142 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 833/1669] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 442 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:45.400275 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc236da5-4f64-4ed7-9120-5bc8372df62a admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:45.580386 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc236da5-4f64-4ed7-9120-5bc8372df62a admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:45.587448 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d4744-9d8a-4226-8bbd-b21a19874132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:45.588262 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d4744-9d8a-4226-8bbd-b21a19874132 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:45.590499 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d4744-9d8a-4226-8bbd-b21a19874132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:45.590834 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d4744-9d8a-4226-8bbd-b21a19874132 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:45.596974 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d4744-9d8a-4226-8bbd-b21a19874132 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:45.600524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d4744-9d8a-4226-8bbd-b21a19874132 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:45.678716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 837/1670] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:11:45 2022] POST /identity/v3/projects => generated 373 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:11:45.682236 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997546de-6a88-425a-a590-f893ac39c5a2 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:45.708998 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997546de-6a88-425a-a590-f893ac39c5a2 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:45.741699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:45.925810 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d92fd-b12f-4a20-9647-3fb02329face admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:45.927208 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d92fd-b12f-4a20-9647-3fb02329face admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:45.930242 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d92fd-b12f-4a20-9647-3fb02329face admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:45.930896 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d92fd-b12f-4a20-9647-3fb02329face admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:45.932350 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5d92fd-b12f-4a20-9647-3fb02329face admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:45.938094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5d92fd-b12f-4a20-9647-3fb02329face admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:46.005064 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 834/1671] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:11:45 2022] POST /identity/v3/users => generated 293 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:11:46.011140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d4744-9d8a-4226-8bbd-b21a19874132 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:46.044566 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d4744-9d8a-4226-8bbd-b21a19874132 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:46.053973 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc3918-0947-4e93-bf0d-94c95e1466e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:46.054756 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc3918-0947-4e93-bf0d-94c95e1466e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:46.055043 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc3918-0947-4e93-bf0d-94c95e1466e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:46.055378 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc3918-0947-4e93-bf0d-94c95e1466e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:46.056211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc3918-0947-4e93-bf0d-94c95e1466e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:46.059420 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc3918-0947-4e93-bf0d-94c95e1466e0 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:46.076889 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 838/1672] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:46.083158 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d92fd-b12f-4a20-9647-3fb02329face admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:46.131489 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d92fd-b12f-4a20-9647-3fb02329face admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:46.150668 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911cf78-5135-48e1-bd12-e7b92321fbf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:46.153094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911cf78-5135-48e1-bd12-e7b92321fbf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:46.155956 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911cf78-5135-48e1-bd12-e7b92321fbf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:46.156403 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911cf78-5135-48e1-bd12-e7b92321fbf0 admin admin] PATH_INFO: `/v3/projects/ee23adadf6d7433f85aa470ca30e49ea/users/bef8db3b02a2403795cca40596c00e89/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:46.252138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911cf78-5135-48e1-bd12-e7b92321fbf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee23adadf6d7433f85aa470ca30e49ea, user_id=bef8db3b02a2403795cca40596c00e89, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:46.263120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911cf78-5135-48e1-bd12-e7b92321fbf0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:46.302259 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 835/1673] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:11:46 2022] PUT /identity/v3/projects/ee23adadf6d7433f85aa470ca30e49ea/users/bef8db3b02a2403795cca40596c00e89/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:11:46.305587 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc3918-0947-4e93-bf0d-94c95e1466e0 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:46.337673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc3918-0947-4e93-bf0d-94c95e1466e0 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:46.591607 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b6d01-eb4f-411a-9d9d-cb3b739d4b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:46.593012 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b6d01-eb4f-411a-9d9d-cb3b739d4b0b admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:46.593956 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b6d01-eb4f-411a-9d9d-cb3b739d4b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:46.594591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b6d01-eb4f-411a-9d9d-cb3b739d4b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:46.604201 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972b6d01-eb4f-411a-9d9d-cb3b739d4b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:46.611927 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972b6d01-eb4f-411a-9d9d-cb3b739d4b0b admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:46.622538 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911cf78-5135-48e1-bd12-e7b92321fbf0 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:46.626202 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1911cf78-5135-48e1-bd12-e7b92321fbf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:46.652457 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911cf78-5135-48e1-bd12-e7b92321fbf0 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:46.661002 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 839/1674] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:46.669738 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ea520-67f8-44c3-857f-e0161c5a1e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:46.671894 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ea520-67f8-44c3-857f-e0161c5a1e21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:46.672195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ea520-67f8-44c3-857f-e0161c5a1e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:46.672591 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ea520-67f8-44c3-857f-e0161c5a1e21 admin admin] PATH_INFO: `/v3/projects/fb9a0fb7ff804caba3b1a6c4cdf7779a` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:46.675385 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1ea520-67f8-44c3-857f-e0161c5a1e21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb9a0fb7ff804caba3b1a6c4cdf7779a)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:46.677033 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b6d01-eb4f-411a-9d9d-cb3b739d4b0b admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:46.679051 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1ea520-67f8-44c3-857f-e0161c5a1e21 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:46.740252 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b6d01-eb4f-411a-9d9d-cb3b739d4b0b admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:46.762615 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dba7c6-0072-4cc6-81fd-bac8785d2314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:46.763594 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dba7c6-0072-4cc6-81fd-bac8785d2314 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:46.764621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dba7c6-0072-4cc6-81fd-bac8785d2314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:46.767107 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dba7c6-0072-4cc6-81fd-bac8785d2314 admin admin] PATH_INFO: `/v3/projects/ee23adadf6d7433f85aa470ca30e49ea/users/bef8db3b02a2403795cca40596c00e89/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:46.771777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dba7c6-0072-4cc6-81fd-bac8785d2314 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee23adadf6d7433f85aa470ca30e49ea, user_id=bef8db3b02a2403795cca40596c00e89, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:46.774818 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dba7c6-0072-4cc6-81fd-bac8785d2314 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:46.789069 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1c1ea520-67f8-44c3-857f-e0161c5a1e21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb9a0fb7ff804caba3b1a6c4cdf7779a', 'request_id': 'req-1c1ea520-67f8-44c3-857f-e0161c5a1e21'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:46.811304 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 840/1675] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:11:46 2022] PUT /identity/v3/projects/ee23adadf6d7433f85aa470ca30e49ea/users/bef8db3b02a2403795cca40596c00e89/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:11:46.812189 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1c1ea520-67f8-44c3-857f-e0161c5a1e21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb9a0fb7ff804caba3b1a6c4cdf7779a', 'request_id': 'req-1c1ea520-67f8-44c3-857f-e0161c5a1e21'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:46.831583 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1c1ea520-67f8-44c3-857f-e0161c5a1e21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb9a0fb7ff804caba3b1a6c4cdf7779a', 'request_id': 'req-1c1ea520-67f8-44c3-857f-e0161c5a1e21'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:46.831685 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a85ec-5584-4116-8cf7-4ff9d0d095fb None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:46.831915 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a85ec-5584-4116-8cf7-4ff9d0d095fb None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:46.832203 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a85ec-5584-4116-8cf7-4ff9d0d095fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:46.873157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1c1ea520-67f8-44c3-857f-e0161c5a1e21 admin admin] The token cache is being invalidate because project fb9a0fb7ff804caba3b1a6c4cdf7779a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:11:46.873982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1c1ea520-67f8-44c3-857f-e0161c5a1e21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:46.874976 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1c1ea520-67f8-44c3-857f-e0161c5a1e21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:46.881074 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 836/1676] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:11:46 2022] DELETE /identity/v3/projects/fb9a0fb7ff804caba3b1a6c4cdf7779a => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:11:46.883213 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f105b1-b63c-4ca2-b585-a750ceac7bc8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:46.883652 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f105b1-b63c-4ca2-b585-a750ceac7bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:46.884667 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f105b1-b63c-4ca2-b585-a750ceac7bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:46.898459 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-133a85ec-5584-4116-8cf7-4ff9d0d095fb None None] MFA Rules not processed for user `bef8db3b02a2403795cca40596c00e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:46.900791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-133a85ec-5584-4116-8cf7-4ff9d0d095fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:46.917761 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:46.981581 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b0f105b1-b63c-4ca2-b585-a750ceac7bc8 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:46.987924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b0f105b1-b63c-4ca2-b585-a750ceac7bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:47.213643 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 841/1677] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:46 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:47.245216 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a85ec-5584-4116-8cf7-4ff9d0d095fb None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:47.247020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-133a85ec-5584-4116-8cf7-4ff9d0d095fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:47.362100 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 837/1678] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 480 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:47.372347 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e7e08-4e02-4420-9ab7-711d200069f8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:47.372825 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e7e08-4e02-4420-9ab7-711d200069f8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:47.374321 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e7e08-4e02-4420-9ab7-711d200069f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:47.468692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:47.472617 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a85ec-5584-4116-8cf7-4ff9d0d095fb None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:47.491245 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1e4e7e08-4e02-4420-9ab7-711d200069f8 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:47.492126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97493303-84f8-4109-86ec-c80c018fc986 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:47.493715 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97493303-84f8-4109-86ec-c80c018fc986 service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:47.494797 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97493303-84f8-4109-86ec-c80c018fc986 service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:47.496228 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97493303-84f8-4109-86ec-c80c018fc986 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:47.503337 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1e4e7e08-4e02-4420-9ab7-711d200069f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:47.538937 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97493303-84f8-4109-86ec-c80c018fc986 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:47.542880 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97493303-84f8-4109-86ec-c80c018fc986 service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:47.567312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 838/1679] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:11:47.569307 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e7e08-4e02-4420-9ab7-711d200069f8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:47.599849 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 842/1680] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:11:47 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:11:47.625391 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e7e08-4e02-4420-9ab7-711d200069f8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:47.646253 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9625579-45d0-4a2d-978c-8454e7f5bc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:47.647628 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9625579-45d0-4a2d-978c-8454e7f5bc73 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:47.648158 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9625579-45d0-4a2d-978c-8454e7f5bc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:47.648737 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9625579-45d0-4a2d-978c-8454e7f5bc73 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:47.652523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9625579-45d0-4a2d-978c-8454e7f5bc73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:11:47.653144 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9625579-45d0-4a2d-978c-8454e7f5bc73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:47.668452 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9625579-45d0-4a2d-978c-8454e7f5bc73 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:47.785117 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 839/1681] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:11:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:11:47.791092 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97493303-84f8-4109-86ec-c80c018fc986 service cinder] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:47.814672 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97493303-84f8-4109-86ec-c80c018fc986 service cinder] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:47.835986 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec53d95-3c28-4c85-9666-ff9934db9d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:47.837125 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec53d95-3c28-4c85-9666-ff9934db9d5e admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:47.837654 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec53d95-3c28-4c85-9666-ff9934db9d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:47.841537 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec53d95-3c28-4c85-9666-ff9934db9d5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:47.843522 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec53d95-3c28-4c85-9666-ff9934db9d5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:47.848322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec53d95-3c28-4c85-9666-ff9934db9d5e admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:47.929312 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 843/1682] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:11:47 2022] POST /identity/v3/projects => generated 363 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:11:47.930656 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9625579-45d0-4a2d-978c-8454e7f5bc73 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:47.946405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9625579-45d0-4a2d-978c-8454e7f5bc73 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:48.203891 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099be33-87c9-416c-89d9-d8df5085737e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:48.205325 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099be33-87c9-416c-89d9-d8df5085737e admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:48.206164 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099be33-87c9-416c-89d9-d8df5085737e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:48.206916 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099be33-87c9-416c-89d9-d8df5085737e admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:48.208430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099be33-87c9-416c-89d9-d8df5085737e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:48.212685 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099be33-87c9-416c-89d9-d8df5085737e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:48.325785 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 840/1683] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:11:47 2022] POST /identity/v3/users => generated 282 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:11:48.348719 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec53d95-3c28-4c85-9666-ff9934db9d5e admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:48.411909 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec53d95-3c28-4c85-9666-ff9934db9d5e admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:48.445081 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c81301-ca67-4cd6-b68a-84c48d65e6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:48.446078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c81301-ca67-4cd6-b68a-84c48d65e6fa admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:48.446502 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c81301-ca67-4cd6-b68a-84c48d65e6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:48.446956 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c81301-ca67-4cd6-b68a-84c48d65e6fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:48.448182 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c81301-ca67-4cd6-b68a-84c48d65e6fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:48.471220 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c81301-ca67-4cd6-b68a-84c48d65e6fa admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:48.515983 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 844/1684] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:48.518645 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099be33-87c9-416c-89d9-d8df5085737e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:48.554109 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099be33-87c9-416c-89d9-d8df5085737e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:48.594660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c46c16-3a74-4acb-a4bb-d7d1a8eabd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:48.595687 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c46c16-3a74-4acb-a4bb-d7d1a8eabd2b admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:48.596763 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c46c16-3a74-4acb-a4bb-d7d1a8eabd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:48.598380 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c46c16-3a74-4acb-a4bb-d7d1a8eabd2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:48.600359 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c46c16-3a74-4acb-a4bb-d7d1a8eabd2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:48.603232 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c46c16-3a74-4acb-a4bb-d7d1a8eabd2b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:48.666025 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 841/1685] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:11:48.667758 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c81301-ca67-4cd6-b68a-84c48d65e6fa admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:48.690084 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c81301-ca67-4cd6-b68a-84c48d65e6fa admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:48.708897 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483fc64-402a-4428-b851-bd5a4b6675ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:48.709879 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483fc64-402a-4428-b851-bd5a4b6675ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:48.714028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483fc64-402a-4428-b851-bd5a4b6675ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:48.714028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483fc64-402a-4428-b851-bd5a4b6675ec admin admin] PATH_INFO: `/v3/projects/830fb8d7d68d4e49af22dbbeec2e8146/users/e16ef6d8ed18476fa1cad9e9d1c46ea0/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:48.792377 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483fc64-402a-4428-b851-bd5a4b6675ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=830fb8d7d68d4e49af22dbbeec2e8146, user_id=e16ef6d8ed18476fa1cad9e9d1c46ea0, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:48.797261 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483fc64-402a-4428-b851-bd5a4b6675ec admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:48.858635 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 845/1686] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:11:48 2022] PUT /identity/v3/projects/830fb8d7d68d4e49af22dbbeec2e8146/users/e16ef6d8ed18476fa1cad9e9d1c46ea0/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:11:48.870664 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719d51d-2060-420a-a304-7a3e820d31c1 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:48.872351 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719d51d-2060-420a-a304-7a3e820d31c1 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:48.872926 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719d51d-2060-420a-a304-7a3e820d31c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:48.975136 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b719d51d-2060-420a-a304-7a3e820d31c1 None None] MFA Rules not processed for user `e16ef6d8ed18476fa1cad9e9d1c46ea0`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:48.980129 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b719d51d-2060-420a-a304-7a3e820d31c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:49.294771 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 842/1687] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:48 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:11:49.296392 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483fc64-402a-4428-b851-bd5a4b6675ec admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:49.416717 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:49.441670 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483fc64-402a-4428-b851-bd5a4b6675ec admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:49.457991 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3362abb-3654-4847-b175-7da56fc489dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:49.459084 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3362abb-3654-4847-b175-7da56fc489dc admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:49.459615 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3362abb-3654-4847-b175-7da56fc489dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:49.460064 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3362abb-3654-4847-b175-7da56fc489dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:49.460995 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3362abb-3654-4847-b175-7da56fc489dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:49.463961 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3362abb-3654-4847-b175-7da56fc489dc admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:49.527575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719d51d-2060-420a-a304-7a3e820d31c1 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:49.535839 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b719d51d-2060-420a-a304-7a3e820d31c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:49.552803 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 846/1688] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:11:49 2022] POST /identity/v3/projects => generated 365 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:11:49.559154 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3362abb-3654-4847-b175-7da56fc489dc admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:49.585649 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719d51d-2060-420a-a304-7a3e820d31c1 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:49.769802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:49.809865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ab6ac-5d0b-4b36-beae-6a04071a1e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:49.812462 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ab6ac-5d0b-4b36-beae-6a04071a1e64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:49.812462 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ab6ac-5d0b-4b36-beae-6a04071a1e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:49.812990 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ab6ac-5d0b-4b36-beae-6a04071a1e64 admin admin] PATH_INFO: `/v3/users/9528dc0eb5024b659c437751bd5b306b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:49.819177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ab6ac-5d0b-4b36-beae-6a04071a1e64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9528dc0eb5024b659c437751bd5b306b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:49.828454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ab6ac-5d0b-4b36-beae-6a04071a1e64 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:49.861762 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3362abb-3654-4847-b175-7da56fc489dc admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:49.885665 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb716a-2063-4878-b81a-3110714955ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:49.887219 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb716a-2063-4878-b81a-3110714955ce admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:49.891298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb716a-2063-4878-b81a-3110714955ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:49.892129 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb716a-2063-4878-b81a-3110714955ce admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:49.896106 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fb716a-2063-4878-b81a-3110714955ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:49.905046 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fb716a-2063-4878-b81a-3110714955ce admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:49.978786 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c10ab6ac-5d0b-4b36-beae-6a04071a1e64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9528dc0eb5024b659c437751bd5b306b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:49.981021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 847/1689] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:11:49 2022] POST /identity/v3/users => generated 289 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:11:49.987436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb716a-2063-4878-b81a-3110714955ce admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:50.028283 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c10ab6ac-5d0b-4b36-beae-6a04071a1e64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9528dc0eb5024b659c437751bd5b306b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:50.038743 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb716a-2063-4878-b81a-3110714955ce admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:50.049762 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c10ab6ac-5d0b-4b36-beae-6a04071a1e64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9528dc0eb5024b659c437751bd5b306b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:50.068934 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab85597-f672-482d-bd48-708f7ae0606e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:50.070461 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab85597-f672-482d-bd48-708f7ae0606e admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:50.071002 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab85597-f672-482d-bd48-708f7ae0606e admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:50.071752 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab85597-f672-482d-bd48-708f7ae0606e admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:50.073244 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab85597-f672-482d-bd48-708f7ae0606e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:50.080916 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab85597-f672-482d-bd48-708f7ae0606e admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:50.083292 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c10ab6ac-5d0b-4b36-beae-6a04071a1e64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9528dc0eb5024b659c437751bd5b306b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:50.087319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c10ab6ac-5d0b-4b36-beae-6a04071a1e64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9528dc0eb5024b659c437751bd5b306b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:50.110112 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 848/1690] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:49 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:50.115683 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab85597-f672-482d-bd48-708f7ae0606e admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:50.121796 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-6ab85597-f672-482d-bd48-708f7ae0606e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:50.134598 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 843/1691] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:11:49 2022] DELETE /identity/v3/users/9528dc0eb5024b659c437751bd5b306b => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:11:50.141345 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c227e0-40be-4c92-84fd-0ef100ba19cc None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:50.141869 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c227e0-40be-4c92-84fd-0ef100ba19cc None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:50.142327 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c227e0-40be-4c92-84fd-0ef100ba19cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:50.198566 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab85597-f672-482d-bd48-708f7ae0606e admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:50.233659 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-67c227e0-40be-4c92-84fd-0ef100ba19cc None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:50.235003 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-67c227e0-40be-4c92-84fd-0ef100ba19cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:50.407516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630164e4-c863-4878-a424-929786d52067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:50.408712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630164e4-c863-4878-a424-929786d52067 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:50.409037 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630164e4-c863-4878-a424-929786d52067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:50.409446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630164e4-c863-4878-a424-929786d52067 admin admin] PATH_INFO: `/v3/projects/51ddf36d52354a0b94334f62d5cea431/users/73a0706e235346c8a860cb0c0f07f027/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:50.445734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630164e4-c863-4878-a424-929786d52067 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51ddf36d52354a0b94334f62d5cea431, user_id=73a0706e235346c8a860cb0c0f07f027, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:50.450847 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630164e4-c863-4878-a424-929786d52067 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:50.514420 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 849/1692] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:11:50 2022] PUT /identity/v3/projects/51ddf36d52354a0b94334f62d5cea431/users/73a0706e235346c8a860cb0c0f07f027/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:11:50.524927 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 844/1693] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:50.534013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630164e4-c863-4878-a424-929786d52067 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:50.572073 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c227e0-40be-4c92-84fd-0ef100ba19cc None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:50.574110 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-67c227e0-40be-4c92-84fd-0ef100ba19cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:50.722958 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:50.914618 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c227e0-40be-4c92-84fd-0ef100ba19cc None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:50.921662 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630164e4-c863-4878-a424-929786d52067 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:50.956034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e49e8-e312-4b83-afed-9f081f99ac21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:50.972684 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e49e8-e312-4b83-afed-9f081f99ac21 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:50.974833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e49e8-e312-4b83-afed-9f081f99ac21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:50.975432 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e49e8-e312-4b83-afed-9f081f99ac21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:50.983345 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e49e8-e312-4b83-afed-9f081f99ac21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:50.988675 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e49e8-e312-4b83-afed-9f081f99ac21 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:51.047057 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 850/1694] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 515 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:11:51.053876 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e49e8-e312-4b83-afed-9f081f99ac21 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:51.109978 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:51.114627 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e49e8-e312-4b83-afed-9f081f99ac21 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:51.141544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc7012-ee9e-46e1-b0e8-a73cea3a3f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:51.143452 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc7012-ee9e-46e1-b0e8-a73cea3a3f84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:51.143794 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc7012-ee9e-46e1-b0e8-a73cea3a3f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:51.144758 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc7012-ee9e-46e1-b0e8-a73cea3a3f84 admin admin] PATH_INFO: `/v3/projects/51ddf36d52354a0b94334f62d5cea431/users/73a0706e235346c8a860cb0c0f07f027/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:51.155021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc7012-ee9e-46e1-b0e8-a73cea3a3f84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51ddf36d52354a0b94334f62d5cea431, user_id=73a0706e235346c8a860cb0c0f07f027, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:51.160713 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc7012-ee9e-46e1-b0e8-a73cea3a3f84 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:51.241416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 851/1695] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:11:51 2022] PUT /identity/v3/projects/51ddf36d52354a0b94334f62d5cea431/users/73a0706e235346c8a860cb0c0f07f027/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:11:51.243404 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e67b6d-8000-4af4-88e0-886a1481c7ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:51.244456 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3eaab9-d51e-48ac-859e-49bcfb956475 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:51.244575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e67b6d-8000-4af4-88e0-886a1481c7ca service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:51.244952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3eaab9-d51e-48ac-859e-49bcfb956475 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:51.245053 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e67b6d-8000-4af4-88e0-886a1481c7ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:51.245732 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e67b6d-8000-4af4-88e0-886a1481c7ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:51.246778 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3eaab9-d51e-48ac-859e-49bcfb956475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:51.296519 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e67b6d-8000-4af4-88e0-886a1481c7ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:51.303419 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e67b6d-8000-4af4-88e0-886a1481c7ca service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:51.358900 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 845/1696] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:11:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 787 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:11:51.373738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-6a3eaab9-d51e-48ac-859e-49bcfb956475 None None] MFA Rules not processed for user `73a0706e235346c8a860cb0c0f07f027`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:51.375896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-6a3eaab9-d51e-48ac-859e-49bcfb956475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:51.618362 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 852/1697] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:51 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:11:51.663378 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e67b6d-8000-4af4-88e0-886a1481c7ca service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:51.665584 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-81e67b6d-8000-4af4-88e0-886a1481c7ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:51.866705 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e67b6d-8000-4af4-88e0-886a1481c7ca service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:52.039245 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b552a8c-1666-499e-b875-83d65cce12f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:52.040473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b552a8c-1666-499e-b875-83d65cce12f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:52.041091 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b552a8c-1666-499e-b875-83d65cce12f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:52.041742 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b552a8c-1666-499e-b875-83d65cce12f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:52.073417 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b552a8c-1666-499e-b875-83d65cce12f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:52.082083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b552a8c-1666-499e-b875-83d65cce12f0 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:52.121169 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 846/1698] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:11:51 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:11:52.434955 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3eaab9-d51e-48ac-859e-49bcfb956475 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:52.469828 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3eaab9-d51e-48ac-859e-49bcfb956475 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:52.479124 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c5a2a-7da0-4ac6-a872-815ddcf08369 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:52.483413 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c5a2a-7da0-4ac6-a872-815ddcf08369 service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:52.484490 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c5a2a-7da0-4ac6-a872-815ddcf08369 service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:52.485238 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c5a2a-7da0-4ac6-a872-815ddcf08369 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:52.492355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-108c5a2a-7da0-4ac6-a872-815ddcf08369 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:52.734318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108c5a2a-7da0-4ac6-a872-815ddcf08369 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:52.737405 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108c5a2a-7da0-4ac6-a872-815ddcf08369 service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:52.777004 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 853/1699] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:11:52 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:11:53.599356 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb0099-1775-4827-985f-bbd7a0e341d0 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:53.599821 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb0099-1775-4827-985f-bbd7a0e341d0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:53.599919 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb0099-1775-4827-985f-bbd7a0e341d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:53.688600 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-98bb0099-1775-4827-985f-bbd7a0e341d0 None None] MFA Rules not processed for user `515114f0a842463fb2d1265ee33db8eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:53.693445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-98bb0099-1775-4827-985f-bbd7a0e341d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:53.900744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 847/1700] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:11:53.915386 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122789a-410b-4f69-907b-7c1f48c0c1b2 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:53.915905 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122789a-410b-4f69-907b-7c1f48c0c1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:53.916362 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122789a-410b-4f69-907b-7c1f48c0c1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:53.953064 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-7122789a-410b-4f69-907b-7c1f48c0c1b2 None None] MFA Rules not processed for user `515114f0a842463fb2d1265ee33db8eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:53.955629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-7122789a-410b-4f69-907b-7c1f48c0c1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:53.978504 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 854/1701] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:54.015408 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bb0099-1775-4827-985f-bbd7a0e341d0 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:54.019106 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-98bb0099-1775-4827-985f-bbd7a0e341d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:54.175383 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:54.332586 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bb0099-1775-4827-985f-bbd7a0e341d0 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:54.339150 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1692657-cf81-4876-8694-8b986a1c8d1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:54.339928 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1692657-cf81-4876-8694-8b986a1c8d1e service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:54.340219 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1692657-cf81-4876-8694-8b986a1c8d1e service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:54.340681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1692657-cf81-4876-8694-8b986a1c8d1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:54.352366 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1692657-cf81-4876-8694-8b986a1c8d1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:54.354442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1692657-cf81-4876-8694-8b986a1c8d1e service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:54.367364 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 848/1702] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:11:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:11:54.590917 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384d012-8753-422e-acaf-709cd7c3b9d1 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:54.591220 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384d012-8753-422e-acaf-709cd7c3b9d1 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:54.591556 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384d012-8753-422e-acaf-709cd7c3b9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:54.661060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-2384d012-8753-422e-acaf-709cd7c3b9d1 None None] MFA Rules not processed for user `742ae6608b514ebe83d21a936266848a`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:54.662286 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-2384d012-8753-422e-acaf-709cd7c3b9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:54.700589 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:54.802896 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1692657-cf81-4876-8694-8b986a1c8d1e service nova] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:54.806171 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e1692657-cf81-4876-8694-8b986a1c8d1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:54.869080 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 855/1703] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:54 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:55.138042 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1692657-cf81-4876-8694-8b986a1c8d1e service nova] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:55.159729 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0654305-d50a-4e53-b2b9-9a17f22577d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:55.161689 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0654305-d50a-4e53-b2b9-9a17f22577d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:55.164573 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0654305-d50a-4e53-b2b9-9a17f22577d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:55.165580 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0654305-d50a-4e53-b2b9-9a17f22577d3 admin admin] PATH_INFO: `/v3/projects/aefe3b0db19c4d88a4559ea34ba3b7e0` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:55.168326 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0654305-d50a-4e53-b2b9-9a17f22577d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aefe3b0db19c4d88a4559ea34ba3b7e0)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:55.171996 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0654305-d50a-4e53-b2b9-9a17f22577d3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:55.377243 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a0654305-d50a-4e53-b2b9-9a17f22577d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aefe3b0db19c4d88a4559ea34ba3b7e0', 'request_id': 'req-a0654305-d50a-4e53-b2b9-9a17f22577d3'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:55.396461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a0654305-d50a-4e53-b2b9-9a17f22577d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aefe3b0db19c4d88a4559ea34ba3b7e0', 'request_id': 'req-a0654305-d50a-4e53-b2b9-9a17f22577d3'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:55.416634 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a0654305-d50a-4e53-b2b9-9a17f22577d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aefe3b0db19c4d88a4559ea34ba3b7e0', 'request_id': 'req-a0654305-d50a-4e53-b2b9-9a17f22577d3'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:55.447637 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a0654305-d50a-4e53-b2b9-9a17f22577d3 admin admin] The token cache is being invalidate because project aefe3b0db19c4d88a4559ea34ba3b7e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:11:55.449483 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a0654305-d50a-4e53-b2b9-9a17f22577d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:55.450713 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a0654305-d50a-4e53-b2b9-9a17f22577d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:55.458199 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 849/1704] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:11:54 2022] DELETE /identity/v3/projects/aefe3b0db19c4d88a4559ea34ba3b7e0 => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:11:55.459629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384d012-8753-422e-acaf-709cd7c3b9d1 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:55.494624 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384d012-8753-422e-acaf-709cd7c3b9d1 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:55.538900 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0654305-d50a-4e53-b2b9-9a17f22577d3 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:55.542461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a0654305-d50a-4e53-b2b9-9a17f22577d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:55.614474 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:55.652665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0654305-d50a-4e53-b2b9-9a17f22577d3 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:55.663146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6fdc4b-133c-4ca1-8be2-58728188e350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:55.664514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6fdc4b-133c-4ca1-8be2-58728188e350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:55.665155 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6fdc4b-133c-4ca1-8be2-58728188e350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:55.665535 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6fdc4b-133c-4ca1-8be2-58728188e350 admin admin] PATH_INFO: `/v3/users/cafedd57e92740f7bbfaf5f6df41d71e` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:55.667380 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6fdc4b-133c-4ca1-8be2-58728188e350 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cafedd57e92740f7bbfaf5f6df41d71e)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:55.673744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6fdc4b-133c-4ca1-8be2-58728188e350 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:55.808083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0a6fdc4b-133c-4ca1-8be2-58728188e350 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cafedd57e92740f7bbfaf5f6df41d71e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:55.811361 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f82f68-e5eb-4131-ba9c-39f867686a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:55.812383 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f82f68-e5eb-4131-ba9c-39f867686a2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:55.812895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f82f68-e5eb-4131-ba9c-39f867686a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:55.813344 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f82f68-e5eb-4131-ba9c-39f867686a2b admin admin] PATH_INFO: `/v3/users/bef8db3b02a2403795cca40596c00e89` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:55.817731 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f82f68-e5eb-4131-ba9c-39f867686a2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bef8db3b02a2403795cca40596c00e89)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:55.822924 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f82f68-e5eb-4131-ba9c-39f867686a2b admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:55.859593 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0a6fdc4b-133c-4ca1-8be2-58728188e350 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cafedd57e92740f7bbfaf5f6df41d71e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:55.890462 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0a6fdc4b-133c-4ca1-8be2-58728188e350 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cafedd57e92740f7bbfaf5f6df41d71e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:55.905260 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0a6fdc4b-133c-4ca1-8be2-58728188e350 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cafedd57e92740f7bbfaf5f6df41d71e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:55.915866 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0a6fdc4b-133c-4ca1-8be2-58728188e350 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cafedd57e92740f7bbfaf5f6df41d71e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:55.937897 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 850/1705] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:11:55 2022] DELETE /identity/v3/users/cafedd57e92740f7bbfaf5f6df41d71e => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:11:55.944452 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446854a7-ebc1-4696-b540-cc187c7f36ab None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:55.946043 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446854a7-ebc1-4696-b540-cc187c7f36ab None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:55.946550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446854a7-ebc1-4696-b540-cc187c7f36ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:55.954676 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-37f82f68-e5eb-4131-ba9c-39f867686a2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bef8db3b02a2403795cca40596c00e89'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:55.974666 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-37f82f68-e5eb-4131-ba9c-39f867686a2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bef8db3b02a2403795cca40596c00e89'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:56.006015 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-446854a7-ebc1-4696-b540-cc187c7f36ab None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:56.008744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-446854a7-ebc1-4696-b540-cc187c7f36ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:56.014710 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-37f82f68-e5eb-4131-ba9c-39f867686a2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bef8db3b02a2403795cca40596c00e89'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:56.016188 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-37f82f68-e5eb-4131-ba9c-39f867686a2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bef8db3b02a2403795cca40596c00e89'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:56.033497 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-37f82f68-e5eb-4131-ba9c-39f867686a2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bef8db3b02a2403795cca40596c00e89'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:11:56.046201 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 856/1706] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:11:55 2022] DELETE /identity/v3/users/bef8db3b02a2403795cca40596c00e89 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:11:56.231306 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 851/1707] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:56.277526 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f82f68-e5eb-4131-ba9c-39f867686a2b admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:56.295824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-37f82f68-e5eb-4131-ba9c-39f867686a2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:56.413138 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:56.461895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f82f68-e5eb-4131-ba9c-39f867686a2b admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:56.477715 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305f4d0-6ca8-4857-9a2b-ee10d6cb1c56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:56.478813 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305f4d0-6ca8-4857-9a2b-ee10d6cb1c56 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:56.479631 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305f4d0-6ca8-4857-9a2b-ee10d6cb1c56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:56.480263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305f4d0-6ca8-4857-9a2b-ee10d6cb1c56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:56.614164 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd3bd0-c394-41ba-8eb3-1250b83ffc20 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:56.614462 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd3bd0-c394-41ba-8eb3-1250b83ffc20 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:56.614799 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd3bd0-c394-41ba-8eb3-1250b83ffc20 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:56.616346 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 852/1708] 10.222.0.17 () {54 vars in 912 bytes} [Thu Jun 16 16:11:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:11:56.636458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe5f0d-1f51-45c3-8670-6d14fe952b69 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:56.637196 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe5f0d-1f51-45c3-8670-6d14fe952b69 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:56.637537 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe5f0d-1f51-45c3-8670-6d14fe952b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:56.656990 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305f4d0-6ca8-4857-9a2b-ee10d6cb1c56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:56.660689 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305f4d0-6ca8-4857-9a2b-ee10d6cb1c56 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:56.685752 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 857/1709] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:11:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:11:56.723198 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-60fe5f0d-1f51-45c3-8670-6d14fe952b69 None None] MFA Rules not processed for user `5a84f8aed50c4b3e8e5b33c5b587b26e`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:56.725047 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-60fe5f0d-1f51-45c3-8670-6d14fe952b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:56.735277 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 853/1710] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 16:11:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:11:56.890562 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305f4d0-6ca8-4857-9a2b-ee10d6cb1c56 service neutron] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:56.892361 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-5305f4d0-6ca8-4857-9a2b-ee10d6cb1c56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:56.916820 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305f4d0-6ca8-4857-9a2b-ee10d6cb1c56 service neutron] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:56.926093 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ea8d7e-d165-4018-bf04-d298009bf7ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:56.928304 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea8d7e-d165-4018-bf04-d298009bf7ac service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:56.928672 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea8d7e-d165-4018-bf04-d298009bf7ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:56.931349 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea8d7e-d165-4018-bf04-d298009bf7ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:56.950092 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ea8d7e-d165-4018-bf04-d298009bf7ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:56.955532 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ea8d7e-d165-4018-bf04-d298009bf7ac service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:56.990987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 858/1711] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:11:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:11:58.853786 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335ee73-2013-4561-a162-e34f424428f9 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:58.854629 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335ee73-2013-4561-a162-e34f424428f9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:58.854629 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335ee73-2013-4561-a162-e34f424428f9 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:58.856174 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 854/1712] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jun 16 16:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:11:58.955984 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe060c-f136-45b8-94d2-d410721fd996 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:58.956550 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe060c-f136-45b8-94d2-d410721fd996 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:58.956550 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe060c-f136-45b8-94d2-d410721fd996 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:58.958427 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 859/1713] 10.222.0.17 () {54 vars in 960 bytes} [Thu Jun 16 16:11:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:11:58.965048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9745319-e42b-4be2-b750-12d4aec7146c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:58.965742 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9745319-e42b-4be2-b750-12d4aec7146c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:58.966193 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9745319-e42b-4be2-b750-12d4aec7146c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:59.009429 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b9745319-e42b-4be2-b750-12d4aec7146c None None] MFA Rules not processed for user `36b63614297346e29ff276eab444fe5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:59.011814 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b9745319-e42b-4be2-b750-12d4aec7146c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:59.019200 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6cecec-3876-420e-9e88-ea343139b89b None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:59.019498 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6cecec-3876-420e-9e88-ea343139b89b None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:59.019779 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6cecec-3876-420e-9e88-ea343139b89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:59.058136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-8a6cecec-3876-420e-9e88-ea343139b89b None None] MFA Rules not processed for user `36b63614297346e29ff276eab444fe5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:59.059389 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-8a6cecec-3876-420e-9e88-ea343139b89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:59.128027 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:11:59.224237 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 860/1714] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jun 16 16:11:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:59.242565 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 855/1715] 10.222.0.17 () {58 vars in 1057 bytes} [Thu Jun 16 16:11:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:59.245740 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6cecec-3876-420e-9e88-ea343139b89b None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:59.249808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9745319-e42b-4be2-b750-12d4aec7146c None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:59.251408 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b9745319-e42b-4be2-b750-12d4aec7146c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:59.264971 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6cecec-3876-420e-9e88-ea343139b89b None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:59.278777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e8717-1c8f-4a0a-a853-7a555c4a672f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36b63614297346e29ff276eab444fe5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:59.280154 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e8717-1c8f-4a0a-a853-7a555c4a672f service placement] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:59.281228 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e8717-1c8f-4a0a-a853-7a555c4a672f service placement] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:59.281228 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e8717-1c8f-4a0a-a853-7a555c4a672f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:59.288406 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-481e8717-1c8f-4a0a-a853-7a555c4a672f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:59.289314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9745319-e42b-4be2-b750-12d4aec7146c None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:11:59.295833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc381c-7f27-4bf3-aa50-d1e35a6931ca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36b63614297346e29ff276eab444fe5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:11:59.296831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc381c-7f27-4bf3-aa50-d1e35a6931ca service placement] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:59.297171 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc381c-7f27-4bf3-aa50-d1e35a6931ca service placement] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:59.297494 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc381c-7f27-4bf3-aa50-d1e35a6931ca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:59.341823 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e8717-1c8f-4a0a-a853-7a555c4a672f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:59.342243 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc381c-7f27-4bf3-aa50-d1e35a6931ca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:11:59.343670 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e8717-1c8f-4a0a-a853-7a555c4a672f service placement] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:59.343878 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc381c-7f27-4bf3-aa50-d1e35a6931ca service placement] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:11:59.377318 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 856/1716] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:11:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:11:59.378903 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f53ac5-4509-48f4-ab9a-464ff47e8829 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:11:59.379673 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f53ac5-4509-48f4-ab9a-464ff47e8829 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:11:59.379673 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f53ac5-4509-48f4-ab9a-464ff47e8829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:11:59.407836 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 861/1717] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:11:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:11:59.464626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-92f53ac5-4509-48f4-ab9a-464ff47e8829 None None] MFA Rules not processed for user `515114f0a842463fb2d1265ee33db8eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:11:59.467460 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-92f53ac5-4509-48f4-ab9a-464ff47e8829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:11:59.760146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 857/1718] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:11:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:11:59.996133 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e8717-1c8f-4a0a-a853-7a555c4a672f service placement] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:11:59.999347 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-481e8717-1c8f-4a0a-a853-7a555c4a672f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:00.080612 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e8717-1c8f-4a0a-a853-7a555c4a672f service placement] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:00.109360 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019420fa-311a-4758-a6fc-194c8a4d6fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:00.116016 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019420fa-311a-4758-a6fc-194c8a4d6fa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:00.116555 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019420fa-311a-4758-a6fc-194c8a4d6fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:00.121973 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019420fa-311a-4758-a6fc-194c8a4d6fa8 admin admin] PATH_INFO: `/v3/projects/ee23adadf6d7433f85aa470ca30e49ea` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:00.135511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019420fa-311a-4758-a6fc-194c8a4d6fa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee23adadf6d7433f85aa470ca30e49ea)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:00.155083 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019420fa-311a-4758-a6fc-194c8a4d6fa8 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:00.230197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f538c-bc84-4574-9769-ae8332fea35a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:00.231231 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f538c-bc84-4574-9769-ae8332fea35a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:00.231754 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f538c-bc84-4574-9769-ae8332fea35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:00.418442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f13f538c-bc84-4574-9769-ae8332fea35a None None] MFA Rules not processed for user `4d48e41e19584c5c8773c9f445355ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:00.424346 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f13f538c-bc84-4574-9769-ae8332fea35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:00.537499 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-019420fa-311a-4758-a6fc-194c8a4d6fa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee23adadf6d7433f85aa470ca30e49ea', 'request_id': 'req-019420fa-311a-4758-a6fc-194c8a4d6fa8'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:00.549507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:00.569831 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-019420fa-311a-4758-a6fc-194c8a4d6fa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee23adadf6d7433f85aa470ca30e49ea', 'request_id': 'req-019420fa-311a-4758-a6fc-194c8a4d6fa8'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:00.583227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-019420fa-311a-4758-a6fc-194c8a4d6fa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee23adadf6d7433f85aa470ca30e49ea', 'request_id': 'req-019420fa-311a-4758-a6fc-194c8a4d6fa8'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:00.622332 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 858/1719] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:00.623585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f538c-bc84-4574-9769-ae8332fea35a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:00.627747 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f13f538c-bc84-4574-9769-ae8332fea35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:00.639554 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-019420fa-311a-4758-a6fc-194c8a4d6fa8 admin admin] The token cache is being invalidate because project ee23adadf6d7433f85aa470ca30e49ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:00.639854 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-019420fa-311a-4758-a6fc-194c8a4d6fa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:00.644413 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-019420fa-311a-4758-a6fc-194c8a4d6fa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:00.653560 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 862/1720] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:11:59 2022] DELETE /identity/v3/projects/ee23adadf6d7433f85aa470ca30e49ea => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:00.659904 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d01a4-b412-49cc-bec3-419ba3a832d8 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:00.661978 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d01a4-b412-49cc-bec3-419ba3a832d8 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:00.663218 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d01a4-b412-49cc-bec3-419ba3a832d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:00.670453 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f538c-bc84-4574-9769-ae8332fea35a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:00.735334 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-104d01a4-b412-49cc-bec3-419ba3a832d8 None None] MFA Rules not processed for user `4d48e41e19584c5c8773c9f445355ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:00.739358 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-104d01a4-b412-49cc-bec3-419ba3a832d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:00.863285 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:00.876855 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6fc6e2-a22d-40e7-a18c-1985657922ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:00.877838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6fc6e2-a22d-40e7-a18c-1985657922ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:00.879135 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6fc6e2-a22d-40e7-a18c-1985657922ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:00.879491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6fc6e2-a22d-40e7-a18c-1985657922ee admin admin] PATH_INFO: `/v3/projects/e2442bae81a1434fba74abd03bc94eec` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:00.883693 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6fc6e2-a22d-40e7-a18c-1985657922ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2442bae81a1434fba74abd03bc94eec)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:00.890132 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6fc6e2-a22d-40e7-a18c-1985657922ee admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:00.921571 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 863/1721] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:00.924908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d61060-d7f6-43e8-a8e7-4831270c2215 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:00.924908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d61060-d7f6-43e8-a8e7-4831270c2215 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:00.924908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d61060-d7f6-43e8-a8e7-4831270c2215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:00.955279 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-d1d61060-d7f6-43e8-a8e7-4831270c2215 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:00.957465 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-d1d61060-d7f6-43e8-a8e7-4831270c2215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:01.007105 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-be6fc6e2-a22d-40e7-a18c-1985657922ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2442bae81a1434fba74abd03bc94eec', 'request_id': 'req-be6fc6e2-a22d-40e7-a18c-1985657922ee'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:01.017416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 864/1722] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:01.020261 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d61060-d7f6-43e8-a8e7-4831270c2215 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:01.026484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-d1d61060-d7f6-43e8-a8e7-4831270c2215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:01.034681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-be6fc6e2-a22d-40e7-a18c-1985657922ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2442bae81a1434fba74abd03bc94eec', 'request_id': 'req-be6fc6e2-a22d-40e7-a18c-1985657922ee'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:01.057321 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-be6fc6e2-a22d-40e7-a18c-1985657922ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2442bae81a1434fba74abd03bc94eec', 'request_id': 'req-be6fc6e2-a22d-40e7-a18c-1985657922ee'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:01.103404 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-be6fc6e2-a22d-40e7-a18c-1985657922ee admin admin] The token cache is being invalidate because project e2442bae81a1434fba74abd03bc94eec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:01.103763 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-be6fc6e2-a22d-40e7-a18c-1985657922ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:01.114762 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-be6fc6e2-a22d-40e7-a18c-1985657922ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:01.118041 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 859/1723] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:00 2022] DELETE /identity/v3/projects/e2442bae81a1434fba74abd03bc94eec => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:01.120011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2649c-23c5-48e5-82f7-a54926637f1c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:01.120509 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2649c-23c5-48e5-82f7-a54926637f1c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:01.121421 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2649c-23c5-48e5-82f7-a54926637f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:01.193217 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-bba2649c-23c5-48e5-82f7-a54926637f1c None None] MFA Rules not processed for user `80386d7afc994931af05fd49fa8043d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:01.194513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bba2649c-23c5-48e5-82f7-a54926637f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:01.276243 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d61060-d7f6-43e8-a8e7-4831270c2215 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:01.374756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 860/1724] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:01.377243 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510e7c6-0be9-4a64-8a7f-d7d0e4506418 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:01.377515 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510e7c6-0be9-4a64-8a7f-d7d0e4506418 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:01.377866 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510e7c6-0be9-4a64-8a7f-d7d0e4506418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:01.437383 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8510e7c6-0be9-4a64-8a7f-d7d0e4506418 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:01.443623 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8510e7c6-0be9-4a64-8a7f-d7d0e4506418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:01.501483 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12b8d1-b55b-4d34-a2ec-c7808665d3bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:01.502652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12b8d1-b55b-4d34-a2ec-c7808665d3bc service nova] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:01.503140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12b8d1-b55b-4d34-a2ec-c7808665d3bc service nova] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:01.503670 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12b8d1-b55b-4d34-a2ec-c7808665d3bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:01.516770 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-1e12b8d1-b55b-4d34-a2ec-c7808665d3bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:01.567490 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:01.649393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 861/1725] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:01.650937 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8510e7c6-0be9-4a64-8a7f-d7d0e4506418 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:01.662280 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8510e7c6-0be9-4a64-8a7f-d7d0e4506418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:01.707734 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8510e7c6-0be9-4a64-8a7f-d7d0e4506418 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:01.734069 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e12b8d1-b55b-4d34-a2ec-c7808665d3bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:01.735780 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e12b8d1-b55b-4d34-a2ec-c7808665d3bc service nova] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:01.741645 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ed986-3315-456f-8b35-d25bf23e7ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:01.745760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ed986-3315-456f-8b35-d25bf23e7ba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:01.747523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ed986-3315-456f-8b35-d25bf23e7ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:01.747841 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ed986-3315-456f-8b35-d25bf23e7ba5 admin admin] PATH_INFO: `/v3/users/6578c95cfb4349e5be5d9eb6fee7c26a` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:01.752538 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ed986-3315-456f-8b35-d25bf23e7ba5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6578c95cfb4349e5be5d9eb6fee7c26a)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:01.759577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ed986-3315-456f-8b35-d25bf23e7ba5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:01.776237 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 865/1726] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 756 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:01.789386 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12b8d1-b55b-4d34-a2ec-c7808665d3bc service nova] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:01.800866 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:01.830751 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12b8d1-b55b-4d34-a2ec-c7808665d3bc service nova] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:01.843448 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5faf294-c683-42b6-bddf-9cac1f385bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:01.844180 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5faf294-c683-42b6-bddf-9cac1f385bf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:01.844473 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5faf294-c683-42b6-bddf-9cac1f385bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:01.844756 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5faf294-c683-42b6-bddf-9cac1f385bf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:01.845556 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5faf294-c683-42b6-bddf-9cac1f385bf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:12:01.845868 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5faf294-c683-42b6-bddf-9cac1f385bf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:01.847750 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5faf294-c683-42b6-bddf-9cac1f385bf2 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:01.880239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 866/1727] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:12:01.881267 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5faf294-c683-42b6-bddf-9cac1f385bf2 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:01.894716 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a54ed986-3315-456f-8b35-d25bf23e7ba5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6578c95cfb4349e5be5d9eb6fee7c26a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:01.911121 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5faf294-c683-42b6-bddf-9cac1f385bf2 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:01.929969 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140ff82-61b8-4f5d-a696-ca99e9ae7b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:01.930694 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140ff82-61b8-4f5d-a696-ca99e9ae7b89 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:01.930983 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140ff82-61b8-4f5d-a696-ca99e9ae7b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:01.931277 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140ff82-61b8-4f5d-a696-ca99e9ae7b89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:01.931986 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c140ff82-61b8-4f5d-a696-ca99e9ae7b89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:01.934189 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c140ff82-61b8-4f5d-a696-ca99e9ae7b89 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:01.941181 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a54ed986-3315-456f-8b35-d25bf23e7ba5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6578c95cfb4349e5be5d9eb6fee7c26a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:01.958671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a54ed986-3315-456f-8b35-d25bf23e7ba5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6578c95cfb4349e5be5d9eb6fee7c26a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:01.977667 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a54ed986-3315-456f-8b35-d25bf23e7ba5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6578c95cfb4349e5be5d9eb6fee7c26a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:01.979687 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a54ed986-3315-456f-8b35-d25bf23e7ba5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6578c95cfb4349e5be5d9eb6fee7c26a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:02.001276 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 862/1728] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:01 2022] DELETE /identity/v3/users/6578c95cfb4349e5be5d9eb6fee7c26a => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:02.021734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 867/1729] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:01 2022] POST /identity/v3/projects => generated 355 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:02.023035 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ed986-3315-456f-8b35-d25bf23e7ba5 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:02.024500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a54ed986-3315-456f-8b35-d25bf23e7ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:02.178063 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ed986-3315-456f-8b35-d25bf23e7ba5 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:02.318700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff454d55-e86e-453c-91b4-476a1d9640ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:02.319513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff454d55-e86e-453c-91b4-476a1d9640ac admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:02.319796 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff454d55-e86e-453c-91b4-476a1d9640ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:02.320070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff454d55-e86e-453c-91b4-476a1d9640ac admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:02.320784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff454d55-e86e-453c-91b4-476a1d9640ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:02.331597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff454d55-e86e-453c-91b4-476a1d9640ac admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:02.374578 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 863/1730] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:02 2022] POST /identity/v3/users => generated 278 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:02.383742 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140ff82-61b8-4f5d-a696-ca99e9ae7b89 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:02.403508 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140ff82-61b8-4f5d-a696-ca99e9ae7b89 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:02.422576 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14784567-500a-49c1-a5e9-e352c8b6dc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:02.424065 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14784567-500a-49c1-a5e9-e352c8b6dc4f admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:02.424566 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14784567-500a-49c1-a5e9-e352c8b6dc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:02.425090 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14784567-500a-49c1-a5e9-e352c8b6dc4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:02.428561 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14784567-500a-49c1-a5e9-e352c8b6dc4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:02.430913 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14784567-500a-49c1-a5e9-e352c8b6dc4f admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:02.474249 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 868/1731] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:02.478487 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff454d55-e86e-453c-91b4-476a1d9640ac admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:02.495127 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff454d55-e86e-453c-91b4-476a1d9640ac admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:02.508561 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c69045-e058-4de6-b520-5703c4eccd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:02.509397 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c69045-e058-4de6-b520-5703c4eccd6a admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:02.509684 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c69045-e058-4de6-b520-5703c4eccd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:02.510030 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c69045-e058-4de6-b520-5703c4eccd6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:02.510783 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c69045-e058-4de6-b520-5703c4eccd6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:02.512429 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c69045-e058-4de6-b520-5703c4eccd6a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:02.539849 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 864/1732] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:02.544285 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14784567-500a-49c1-a5e9-e352c8b6dc4f admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:02.560616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14784567-500a-49c1-a5e9-e352c8b6dc4f admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:02.571009 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012be83b-07c8-4d7c-95eb-bf40c09b08e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:02.571942 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012be83b-07c8-4d7c-95eb-bf40c09b08e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:02.572227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012be83b-07c8-4d7c-95eb-bf40c09b08e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:02.572504 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012be83b-07c8-4d7c-95eb-bf40c09b08e8 admin admin] PATH_INFO: `/v3/projects/901c3a6a0a8e4034ad0e23a1b57c7bf5/users/8fb1375fb4a9462eb7b2ec1ae0ff7836/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:02.608434 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012be83b-07c8-4d7c-95eb-bf40c09b08e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=901c3a6a0a8e4034ad0e23a1b57c7bf5, user_id=8fb1375fb4a9462eb7b2ec1ae0ff7836, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:02.618258 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012be83b-07c8-4d7c-95eb-bf40c09b08e8 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:02.641118 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 869/1733] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:02 2022] PUT /identity/v3/projects/901c3a6a0a8e4034ad0e23a1b57c7bf5/users/8fb1375fb4a9462eb7b2ec1ae0ff7836/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:02.656909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1432fe-6f95-4e10-8104-46a510bc6404 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:02.657197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1432fe-6f95-4e10-8104-46a510bc6404 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:02.657477 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1432fe-6f95-4e10-8104-46a510bc6404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:02.708067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:02.718470 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9c1432fe-6f95-4e10-8104-46a510bc6404 None None] MFA Rules not processed for user `8fb1375fb4a9462eb7b2ec1ae0ff7836`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:02.719752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9c1432fe-6f95-4e10-8104-46a510bc6404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:02.830271 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abaddb-dece-4702-960b-9c077a0e960c None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:02.830553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abaddb-dece-4702-960b-9c077a0e960c None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:02.831091 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abaddb-dece-4702-960b-9c077a0e960c None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:02.832744 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 870/1734] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jun 16 16:12:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:12:02.850935 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250b9b0-6d09-48b5-b9c4-9e7d33dc8a9e None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:02.851223 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250b9b0-6d09-48b5-b9c4-9e7d33dc8a9e None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:02.851803 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250b9b0-6d09-48b5-b9c4-9e7d33dc8a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:02.894156 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-6250b9b0-6d09-48b5-b9c4-9e7d33dc8a9e None None] MFA Rules not processed for user `36b63614297346e29ff276eab444fe5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:02.895590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-6250b9b0-6d09-48b5-b9c4-9e7d33dc8a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:02.933327 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 865/1735] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:02 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:02.934631 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1432fe-6f95-4e10-8104-46a510bc6404 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:03.179801 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 871/1736] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jun 16 16:12:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:12:03.180831 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250b9b0-6d09-48b5-b9c4-9e7d33dc8a9e None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:03.182364 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-6250b9b0-6d09-48b5-b9c4-9e7d33dc8a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:03.225501 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1432fe-6f95-4e10-8104-46a510bc6404 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:03.245417 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a30914-3fa0-40d9-8cc2-6cd6cd09fa4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:03.246298 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a30914-3fa0-40d9-8cc2-6cd6cd09fa4b admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:03.251063 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a30914-3fa0-40d9-8cc2-6cd6cd09fa4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:03.251575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a30914-3fa0-40d9-8cc2-6cd6cd09fa4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:03.252391 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a30914-3fa0-40d9-8cc2-6cd6cd09fa4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:03.254511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a30914-3fa0-40d9-8cc2-6cd6cd09fa4b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:03.293142 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 866/1737] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:02 2022] POST /identity/v3/projects => generated 355 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:03.295418 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20058667-0d3c-4e91-a23b-151081b0ed27 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:03.295825 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20058667-0d3c-4e91-a23b-151081b0ed27 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:03.296396 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20058667-0d3c-4e91-a23b-151081b0ed27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:03.315076 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:03.318549 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-20058667-0d3c-4e91-a23b-151081b0ed27 None None] MFA Rules not processed for user `2c2e0e19880f4b048faf034dd38c7365`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:03.320022 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-20058667-0d3c-4e91-a23b-151081b0ed27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:03.389625 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250b9b0-6d09-48b5-b9c4-9e7d33dc8a9e None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:03.420928 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 867/1738] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 16:12:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:03.448059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20058667-0d3c-4e91-a23b-151081b0ed27 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:03.593807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:03.667769 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f43e1-0805-4be6-879c-5990f8ed7410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:03.668728 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f43e1-0805-4be6-879c-5990f8ed7410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:03.676372 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f43e1-0805-4be6-879c-5990f8ed7410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:03.676849 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f43e1-0805-4be6-879c-5990f8ed7410 admin admin] PATH_INFO: `/v3/users/6c8d6307851948b5809b123667be0190` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:03.678302 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f43e1-0805-4be6-879c-5990f8ed7410 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c8d6307851948b5809b123667be0190)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:03.699708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f43e1-0805-4be6-879c-5990f8ed7410 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:03.750716 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20058667-0d3c-4e91-a23b-151081b0ed27 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:03.776630 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de392c87-3fa5-48e3-8426-7960821b4228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:03.781158 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de392c87-3fa5-48e3-8426-7960821b4228 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:03.782469 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de392c87-3fa5-48e3-8426-7960821b4228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:03.782845 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de392c87-3fa5-48e3-8426-7960821b4228 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:03.783774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de392c87-3fa5-48e3-8426-7960821b4228 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:03.805213 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de392c87-3fa5-48e3-8426-7960821b4228 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:03.893477 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 868/1739] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:03 2022] POST /identity/v3/users => generated 284 bytes in 469 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:03.893477 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de392c87-3fa5-48e3-8426-7960821b4228 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:03.901876 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-de392c87-3fa5-48e3-8426-7960821b4228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:03.909176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-790f43e1-0805-4be6-879c-5990f8ed7410 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c8d6307851948b5809b123667be0190'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:03.935380 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-790f43e1-0805-4be6-879c-5990f8ed7410 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c8d6307851948b5809b123667be0190'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:03.980604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-790f43e1-0805-4be6-879c-5990f8ed7410 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c8d6307851948b5809b123667be0190'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:03.983298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-790f43e1-0805-4be6-879c-5990f8ed7410 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c8d6307851948b5809b123667be0190'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:04.002536 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-790f43e1-0805-4be6-879c-5990f8ed7410 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c8d6307851948b5809b123667be0190'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:04.020012 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 872/1740] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:03 2022] DELETE /identity/v3/users/6c8d6307851948b5809b123667be0190 => generated 0 bytes in 840 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:04.029384 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f43e1-0805-4be6-879c-5990f8ed7410 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:04.039427 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-790f43e1-0805-4be6-879c-5990f8ed7410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:04.167945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de392c87-3fa5-48e3-8426-7960821b4228 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:04.309917 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e8ae5-f313-4972-b4fe-5bbc1359bb13 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36b63614297346e29ff276eab444fe5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:04.311125 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e8ae5-f313-4972-b4fe-5bbc1359bb13 service placement] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:04.311533 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e8ae5-f313-4972-b4fe-5bbc1359bb13 service placement] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:04.311822 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e8ae5-f313-4972-b4fe-5bbc1359bb13 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:04.314324 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-168e8ae5-f313-4972-b4fe-5bbc1359bb13 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:04.332884 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e8ae5-f313-4972-b4fe-5bbc1359bb13 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:04.335334 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e8ae5-f313-4972-b4fe-5bbc1359bb13 service placement] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:04.343664 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:04.357757 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f43e1-0805-4be6-879c-5990f8ed7410 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:04.362214 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 869/1741] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:04.363080 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e8ae5-f313-4972-b4fe-5bbc1359bb13 service placement] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:04.364630 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-168e8ae5-f313-4972-b4fe-5bbc1359bb13 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:04.389863 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1236eeb3-0459-4be1-81e3-232036d31caf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:04.393837 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236eeb3-0459-4be1-81e3-232036d31caf service nova] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:04.393837 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236eeb3-0459-4be1-81e3-232036d31caf service nova] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:04.394190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236eeb3-0459-4be1-81e3-232036d31caf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:04.401115 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-1236eeb3-0459-4be1-81e3-232036d31caf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:04.485643 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:04.512913 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e8ae5-f313-4972-b4fe-5bbc1359bb13 service placement] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:04.519833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20b9b0-9543-4e0f-bb1d-73e1de3a062a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0082e37ce54f49f7837601349508c89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:04.520527 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20b9b0-9543-4e0f-bb1d-73e1de3a062a service swift] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:04.520939 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20b9b0-9543-4e0f-bb1d-73e1de3a062a service swift] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:04.521241 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20b9b0-9543-4e0f-bb1d-73e1de3a062a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:04.524776 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bc20b9b0-9543-4e0f-bb1d-73e1de3a062a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:04.604000 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1236eeb3-0459-4be1-81e3-232036d31caf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:04.607891 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1236eeb3-0459-4be1-81e3-232036d31caf service nova] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:04.652743 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 873/1742] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 632 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:04.656121 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1236eeb3-0459-4be1-81e3-232036d31caf service nova] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:04.662231 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-1236eeb3-0459-4be1-81e3-232036d31caf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:04.680692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc20b9b0-9543-4e0f-bb1d-73e1de3a062a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:04.684561 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc20b9b0-9543-4e0f-bb1d-73e1de3a062a service swift] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:04.745324 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 870/1743] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 16:12:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:12:04.747407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38783a0-d2af-4181-8d97-abbde8f228bf None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:04.748719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38783a0-d2af-4181-8d97-abbde8f228bf None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:04.748719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38783a0-d2af-4181-8d97-abbde8f228bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:04.815082 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c38783a0-d2af-4181-8d97-abbde8f228bf None None] MFA Rules not processed for user `2c34f05634d34da29088bcf75069958d`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:04.816578 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c38783a0-d2af-4181-8d97-abbde8f228bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:04.967991 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 871/1744] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:04.986985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1eb068-edd8-4dc9-8816-14ca0a8e5795 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:04.987604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1eb068-edd8-4dc9-8816-14ca0a8e5795 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:04.988968 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1eb068-edd8-4dc9-8816-14ca0a8e5795 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:04.991605 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 872/1745] 10.222.0.17 () {54 vars in 912 bytes} [Thu Jun 16 16:12:04 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:12:04.997566 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1eb068-edd8-4dc9-8816-14ca0a8e5795 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:05.023015 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4f1eb068-edd8-4dc9-8816-14ca0a8e5795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:05.081347 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1236eeb3-0459-4be1-81e3-232036d31caf service nova] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:05.100775 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a4170-e8e9-43b5-98f3-2645b29ba880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:05.103499 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a4170-e8e9-43b5-98f3-2645b29ba880 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:05.108920 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a4170-e8e9-43b5-98f3-2645b29ba880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:05.109848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a4170-e8e9-43b5-98f3-2645b29ba880 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:05.111732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a4170-e8e9-43b5-98f3-2645b29ba880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:05.121141 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a4170-e8e9-43b5-98f3-2645b29ba880 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:05.131960 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1eb068-edd8-4dc9-8816-14ca0a8e5795 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:05.162986 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 874/1746] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 509 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:05.167155 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595a475-bc70-4e4b-be67-ea2ee4df8caa None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:05.167700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c032c-0028-4493-9f19-48154688e98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:05.168789 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c032c-0028-4493-9f19-48154688e98a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:05.168945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c032c-0028-4493-9f19-48154688e98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:05.169186 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c032c-0028-4493-9f19-48154688e98a admin admin] PATH_INFO: `/v3/projects/eca1f139068b4b248291159017100c54` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:05.171941 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595a475-bc70-4e4b-be67-ea2ee4df8caa None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:05.173147 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595a475-bc70-4e4b-be67-ea2ee4df8caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:05.176201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c032c-0028-4493-9f19-48154688e98a admin admin] RBAC: Authorizing `identity:delete_project(project_id=eca1f139068b4b248291159017100c54)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:05.178283 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c032c-0028-4493-9f19-48154688e98a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:05.230147 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-5595a475-bc70-4e4b-be67-ea2ee4df8caa None None] MFA Rules not processed for user `5a84f8aed50c4b3e8e5b33c5b587b26e`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:05.234673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-5595a475-bc70-4e4b-be67-ea2ee4df8caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:05.429161 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 875/1747] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 16:12:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:05.432137 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595a475-bc70-4e4b-be67-ea2ee4df8caa None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:05.465936 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4d7c032c-0028-4493-9f19-48154688e98a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eca1f139068b4b248291159017100c54', 'request_id': 'req-4d7c032c-0028-4493-9f19-48154688e98a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:05.495527 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595a475-bc70-4e4b-be67-ea2ee4df8caa None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:05.505346 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4d7c032c-0028-4493-9f19-48154688e98a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eca1f139068b4b248291159017100c54', 'request_id': 'req-4d7c032c-0028-4493-9f19-48154688e98a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:05.565749 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4d7c032c-0028-4493-9f19-48154688e98a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eca1f139068b4b248291159017100c54', 'request_id': 'req-4d7c032c-0028-4493-9f19-48154688e98a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:05.635321 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4d7c032c-0028-4493-9f19-48154688e98a admin admin] The token cache is being invalidate because project eca1f139068b4b248291159017100c54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:05.635582 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4d7c032c-0028-4493-9f19-48154688e98a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:05.639407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4d7c032c-0028-4493-9f19-48154688e98a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:05.659448 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 873/1748] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:04 2022] DELETE /identity/v3/projects/eca1f139068b4b248291159017100c54 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:05.664734 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c032c-0028-4493-9f19-48154688e98a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:05.675400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4d7c032c-0028-4493-9f19-48154688e98a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:05.759547 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:05.847999 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:05.961259 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52b349-00f1-4075-9de8-c32b4436c649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:05.964083 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52b349-00f1-4075-9de8-c32b4436c649 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:05.971445 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52b349-00f1-4075-9de8-c32b4436c649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:05.976881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52b349-00f1-4075-9de8-c32b4436c649 admin admin] PATH_INFO: `/v3/projects/827c811c01da43459a02f59586b36d44/users/9fcff3007a1948c8bb9b3641e42f7392/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:06.075106 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52b349-00f1-4075-9de8-c32b4436c649 admin admin] RBAC: Authorizing `identity:create_grant(project_id=827c811c01da43459a02f59586b36d44, user_id=9fcff3007a1948c8bb9b3641e42f7392, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:06.083280 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52b349-00f1-4075-9de8-c32b4436c649 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:06.093466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c032c-0028-4493-9f19-48154688e98a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:06.111127 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c8db5a-4cd6-47c6-b07d-e232e7a45b9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:06.112126 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8db5a-4cd6-47c6-b07d-e232e7a45b9c service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:06.112503 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8db5a-4cd6-47c6-b07d-e232e7a45b9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:06.112897 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8db5a-4cd6-47c6-b07d-e232e7a45b9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:06.127885 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-77c8db5a-4cd6-47c6-b07d-e232e7a45b9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:06.141461 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 876/1749] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:05 2022] PUT /identity/v3/projects/827c811c01da43459a02f59586b36d44/users/9fcff3007a1948c8bb9b3641e42f7392/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 708 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:06.148836 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6be523-eac7-48b1-99b8-1bdbd6e809ba None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:06.149136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6be523-eac7-48b1-99b8-1bdbd6e809ba None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:06.149474 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6be523-eac7-48b1-99b8-1bdbd6e809ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:06.220336 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c8db5a-4cd6-47c6-b07d-e232e7a45b9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:06.239485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c8db5a-4cd6-47c6-b07d-e232e7a45b9c service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:06.277850 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-8b6be523-eac7-48b1-99b8-1bdbd6e809ba None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:06.285885 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-8b6be523-eac7-48b1-99b8-1bdbd6e809ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:06.323462 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 874/1750] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 664 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:06.343351 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c8db5a-4cd6-47c6-b07d-e232e7a45b9c service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:06.358137 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-77c8db5a-4cd6-47c6-b07d-e232e7a45b9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:06.652986 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 877/1751] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 502 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:06.652986 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6be523-eac7-48b1-99b8-1bdbd6e809ba None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:06.655899 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-8b6be523-eac7-48b1-99b8-1bdbd6e809ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:06.726594 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6be523-eac7-48b1-99b8-1bdbd6e809ba None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:06.760449 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1f674-697f-4f6b-9cd9-ae91c094c16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:06.761603 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1f674-697f-4f6b-9cd9-ae91c094c16f admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:06.766677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c8db5a-4cd6-47c6-b07d-e232e7a45b9c service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:06.767357 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1f674-697f-4f6b-9cd9-ae91c094c16f admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:06.768601 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1f674-697f-4f6b-9cd9-ae91c094c16f admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:06.771025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1f674-697f-4f6b-9cd9-ae91c094c16f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:06.783170 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1f674-697f-4f6b-9cd9-ae91c094c16f admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:06.894296 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 878/1752] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:06.905563 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93123991-aca1-4c8f-8313-2f2699cfa3b0 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:06.920561 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93123991-aca1-4c8f-8313-2f2699cfa3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:06.921266 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93123991-aca1-4c8f-8313-2f2699cfa3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:07.028404 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-93123991-aca1-4c8f-8313-2f2699cfa3b0 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:07.034255 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-93123991-aca1-4c8f-8313-2f2699cfa3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:07.080475 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 879/1753] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:07.081623 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93123991-aca1-4c8f-8313-2f2699cfa3b0 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:07.101870 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f2705-a413-40e7-851c-8852a49d27f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:07.103191 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f2705-a413-40e7-851c-8852a49d27f7 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:07.103833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f2705-a413-40e7-851c-8852a49d27f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:07.106119 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f2705-a413-40e7-851c-8852a49d27f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:07.121382 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-020f2705-a413-40e7-851c-8852a49d27f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:07.139666 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93123991-aca1-4c8f-8313-2f2699cfa3b0 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:07.183259 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92047b-9e1b-46d3-b4ff-c0d7f3d2fcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:07.186001 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92047b-9e1b-46d3-b4ff-c0d7f3d2fcd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:07.186788 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92047b-9e1b-46d3-b4ff-c0d7f3d2fcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:07.187537 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92047b-9e1b-46d3-b4ff-c0d7f3d2fcd0 admin admin] PATH_INFO: `/v3/projects/827c811c01da43459a02f59586b36d44/users/9fcff3007a1948c8bb9b3641e42f7392/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:07.202319 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92047b-9e1b-46d3-b4ff-c0d7f3d2fcd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=827c811c01da43459a02f59586b36d44, user_id=9fcff3007a1948c8bb9b3641e42f7392, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:07.217084 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92047b-9e1b-46d3-b4ff-c0d7f3d2fcd0 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:07.298717 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 880/1754] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:07 2022] PUT /identity/v3/projects/827c811c01da43459a02f59586b36d44/users/9fcff3007a1948c8bb9b3641e42f7392/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:07.307810 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92047b-9e1b-46d3-b4ff-c0d7f3d2fcd0 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:07.349340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:07.407516 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f2705-a413-40e7-851c-8852a49d27f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:07.409177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f2705-a413-40e7-851c-8852a49d27f7 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:07.474012 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 875/1755] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 1149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:07.477008 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3206c-ac1c-4d86-acc3-a3aad2f7ba5e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:07.477189 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3206c-ac1c-4d86-acc3-a3aad2f7ba5e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:07.477862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3206c-ac1c-4d86-acc3-a3aad2f7ba5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:07.642129 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-99f3206c-ac1c-4d86-acc3-a3aad2f7ba5e None None] MFA Rules not processed for user `9fcff3007a1948c8bb9b3641e42f7392`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:07.645272 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-99f3206c-ac1c-4d86-acc3-a3aad2f7ba5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:07.748875 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92047b-9e1b-46d3-b4ff-c0d7f3d2fcd0 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:07.769698 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c341b-6145-403d-b12b-ec55a338eaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:07.770691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c341b-6145-403d-b12b-ec55a338eaeb admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:07.770984 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c341b-6145-403d-b12b-ec55a338eaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:07.771266 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c341b-6145-403d-b12b-ec55a338eaeb admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:07.772141 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c341b-6145-403d-b12b-ec55a338eaeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:12:07.772459 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c341b-6145-403d-b12b-ec55a338eaeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:07.774258 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c341b-6145-403d-b12b-ec55a338eaeb admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:07.807620 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 881/1756] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 508 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:12:07.827548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c341b-6145-403d-b12b-ec55a338eaeb admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:07.870400 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c341b-6145-403d-b12b-ec55a338eaeb admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:07.881140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07f97bc-e04e-4246-932f-c4905bf5cde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:07.882501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f97bc-e04e-4246-932f-c4905bf5cde0 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:07.883252 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f97bc-e04e-4246-932f-c4905bf5cde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:07.884420 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f97bc-e04e-4246-932f-c4905bf5cde0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:07.889756 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07f97bc-e04e-4246-932f-c4905bf5cde0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:07.893435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07f97bc-e04e-4246-932f-c4905bf5cde0 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:08.010928 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 882/1757] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:07 2022] POST /identity/v3/projects => generated 381 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:08.017893 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07f97bc-e04e-4246-932f-c4905bf5cde0 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:08.107386 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 876/1758] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:07 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 632 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:08.110826 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:08.122836 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3206c-ac1c-4d86-acc3-a3aad2f7ba5e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:08.125797 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-99f3206c-ac1c-4d86-acc3-a3aad2f7ba5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:08.308573 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3206c-ac1c-4d86-acc3-a3aad2f7ba5e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:08.332579 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07f97bc-e04e-4246-932f-c4905bf5cde0 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:08.341490 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c41d24-f108-49da-9439-0b3581e8f578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:08.342633 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c41d24-f108-49da-9439-0b3581e8f578 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:08.343025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c41d24-f108-49da-9439-0b3581e8f578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:08.343456 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c41d24-f108-49da-9439-0b3581e8f578 admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:08.344776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c41d24-f108-49da-9439-0b3581e8f578 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:08.349249 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c41d24-f108-49da-9439-0b3581e8f578 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:08.381298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 883/1759] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:08 2022] POST /identity/v3/users => generated 291 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:08.382478 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c41d24-f108-49da-9439-0b3581e8f578 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:08.384517 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-a8c41d24-f108-49da-9439-0b3581e8f578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:08.403139 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c41d24-f108-49da-9439-0b3581e8f578 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:08.410292 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf51e968-209e-4098-a639-55d66cd70474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:08.411247 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf51e968-209e-4098-a639-55d66cd70474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:08.411649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf51e968-209e-4098-a639-55d66cd70474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:08.412001 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf51e968-209e-4098-a639-55d66cd70474 admin admin] PATH_INFO: `/v3/users/e16ef6d8ed18476fa1cad9e9d1c46ea0` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:08.413689 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf51e968-209e-4098-a639-55d66cd70474 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e16ef6d8ed18476fa1cad9e9d1c46ea0)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:08.416010 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf51e968-209e-4098-a639-55d66cd70474 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:08.566021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-cf51e968-209e-4098-a639-55d66cd70474 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e16ef6d8ed18476fa1cad9e9d1c46ea0'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:08.611127 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-cf51e968-209e-4098-a639-55d66cd70474 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e16ef6d8ed18476fa1cad9e9d1c46ea0'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:08.615534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407b261-5926-4ad7-b128-43c4b135240f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:08.618855 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407b261-5926-4ad7-b128-43c4b135240f service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:08.621222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407b261-5926-4ad7-b128-43c4b135240f service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:08.622424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407b261-5926-4ad7-b128-43c4b135240f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:08.754230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-cf51e968-209e-4098-a639-55d66cd70474 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e16ef6d8ed18476fa1cad9e9d1c46ea0'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:08.760679 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-cf51e968-209e-4098-a639-55d66cd70474 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e16ef6d8ed18476fa1cad9e9d1c46ea0'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:08.797461 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-cf51e968-209e-4098-a639-55d66cd70474 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e16ef6d8ed18476fa1cad9e9d1c46ea0'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:08.839351 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 884/1760] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:08 2022] DELETE /identity/v3/users/e16ef6d8ed18476fa1cad9e9d1c46ea0 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:08.847569 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf51e968-209e-4098-a639-55d66cd70474 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:08.860220 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:08.863619 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-cf51e968-209e-4098-a639-55d66cd70474 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:08.995030 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e407b261-5926-4ad7-b128-43c4b135240f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:09.008972 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e407b261-5926-4ad7-b128-43c4b135240f service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:09.054505 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 877/1761] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:08 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 933 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:09.072808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407b261-5926-4ad7-b128-43c4b135240f service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:09.082880 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e407b261-5926-4ad7-b128-43c4b135240f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:09.202293 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407b261-5926-4ad7-b128-43c4b135240f service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:09.281098 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf51e968-209e-4098-a639-55d66cd70474 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:09.316145 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17f314-88c4-4897-b542-79ff6a32fdc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:09.317013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17f314-88c4-4897-b542-79ff6a32fdc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:09.317294 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17f314-88c4-4897-b542-79ff6a32fdc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:09.317579 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17f314-88c4-4897-b542-79ff6a32fdc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:09.318370 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf17f314-88c4-4897-b542-79ff6a32fdc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:09.320272 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf17f314-88c4-4897-b542-79ff6a32fdc8 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:09.390838 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 885/1762] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 550 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:09.404586 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb54ab2-de7d-4822-8406-d5079489bc7f None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:09.407861 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb54ab2-de7d-4822-8406-d5079489bc7f None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:09.408149 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb54ab2-de7d-4822-8406-d5079489bc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:09.502467 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-dcb54ab2-de7d-4822-8406-d5079489bc7f None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:09.503797 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-dcb54ab2-de7d-4822-8406-d5079489bc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:09.542623 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 886/1763] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:09.546527 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb54ab2-de7d-4822-8406-d5079489bc7f None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:09.552571 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-dcb54ab2-de7d-4822-8406-d5079489bc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:09.588286 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:09.603477 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5996d033-292e-4a87-b48a-88e2866b7eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:09.604507 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996d033-292e-4a87-b48a-88e2866b7eff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:09.604815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996d033-292e-4a87-b48a-88e2866b7eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:09.605111 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996d033-292e-4a87-b48a-88e2866b7eff admin admin] PATH_INFO: `/v3/projects/cbf2b6097fc344fab753b7a12d32f49e` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:09.627794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5996d033-292e-4a87-b48a-88e2866b7eff admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbf2b6097fc344fab753b7a12d32f49e)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:09.635330 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5996d033-292e-4a87-b48a-88e2866b7eff admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:09.868633 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5996d033-292e-4a87-b48a-88e2866b7eff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbf2b6097fc344fab753b7a12d32f49e', 'request_id': 'req-5996d033-292e-4a87-b48a-88e2866b7eff'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:09.899933 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5996d033-292e-4a87-b48a-88e2866b7eff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbf2b6097fc344fab753b7a12d32f49e', 'request_id': 'req-5996d033-292e-4a87-b48a-88e2866b7eff'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:09.917827 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb54ab2-de7d-4822-8406-d5079489bc7f None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:09.922543 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5996d033-292e-4a87-b48a-88e2866b7eff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbf2b6097fc344fab753b7a12d32f49e', 'request_id': 'req-5996d033-292e-4a87-b48a-88e2866b7eff'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:09.970574 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5996d033-292e-4a87-b48a-88e2866b7eff admin admin] The token cache is being invalidate because project cbf2b6097fc344fab753b7a12d32f49e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:09.971076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5996d033-292e-4a87-b48a-88e2866b7eff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:09.980157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5996d033-292e-4a87-b48a-88e2866b7eff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:09.983478 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 878/1764] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:09 2022] DELETE /identity/v3/projects/cbf2b6097fc344fab753b7a12d32f49e => generated 0 bytes in 928 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:09.984077 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5996d033-292e-4a87-b48a-88e2866b7eff admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:09.997186 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5996d033-292e-4a87-b48a-88e2866b7eff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:10.221340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96a2f0-1b8c-4f47-a690-00355db5b701 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:10.223158 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96a2f0-1b8c-4f47-a690-00355db5b701 service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:10.223596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96a2f0-1b8c-4f47-a690-00355db5b701 service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:10.223999 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96a2f0-1b8c-4f47-a690-00355db5b701 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:10.230965 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-bf96a2f0-1b8c-4f47-a690-00355db5b701 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:10.236816 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5996d033-292e-4a87-b48a-88e2866b7eff admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:10.250131 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfbb6e8-76ae-4e88-8a03-7d02eb645bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:10.250845 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbb6e8-76ae-4e88-8a03-7d02eb645bef admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:10.251132 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbb6e8-76ae-4e88-8a03-7d02eb645bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:10.251461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbb6e8-76ae-4e88-8a03-7d02eb645bef admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:10.252325 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfbb6e8-76ae-4e88-8a03-7d02eb645bef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:10.256741 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfbb6e8-76ae-4e88-8a03-7d02eb645bef admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:10.285267 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 879/1765] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:10.287154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfbb6e8-76ae-4e88-8a03-7d02eb645bef admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:10.290631 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7bfbb6e8-76ae-4e88-8a03-7d02eb645bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:10.317334 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:10.333482 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfbb6e8-76ae-4e88-8a03-7d02eb645bef admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:10.353126 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd1231-7723-44ab-8006-b9a0b057aa24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:10.357420 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd1231-7723-44ab-8006-b9a0b057aa24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:10.357871 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd1231-7723-44ab-8006-b9a0b057aa24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:10.358309 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd1231-7723-44ab-8006-b9a0b057aa24 admin admin] PATH_INFO: `/v3/users/742ae6608b514ebe83d21a936266848a` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:10.367486 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd1231-7723-44ab-8006-b9a0b057aa24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=742ae6608b514ebe83d21a936266848a)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:10.369841 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd1231-7723-44ab-8006-b9a0b057aa24 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:10.557408 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96a2f0-1b8c-4f47-a690-00355db5b701 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:10.562205 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96a2f0-1b8c-4f47-a690-00355db5b701 service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:10.603395 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f9bd1231-7723-44ab-8006-b9a0b057aa24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '742ae6608b514ebe83d21a936266848a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:10.621810 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 887/1766] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:09 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 1078 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:10.623287 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96a2f0-1b8c-4f47-a690-00355db5b701 service cinder] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:10.630721 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-bf96a2f0-1b8c-4f47-a690-00355db5b701 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:10.703786 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f9bd1231-7723-44ab-8006-b9a0b057aa24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '742ae6608b514ebe83d21a936266848a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:10.741353 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f9bd1231-7723-44ab-8006-b9a0b057aa24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '742ae6608b514ebe83d21a936266848a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:10.812212 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f9bd1231-7723-44ab-8006-b9a0b057aa24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '742ae6608b514ebe83d21a936266848a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:10.815599 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f9bd1231-7723-44ab-8006-b9a0b057aa24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '742ae6608b514ebe83d21a936266848a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:10.863908 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 880/1767] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:10 2022] DELETE /identity/v3/users/742ae6608b514ebe83d21a936266848a => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:10.864680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd1231-7723-44ab-8006-b9a0b057aa24 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:10.879021 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f9bd1231-7723-44ab-8006-b9a0b057aa24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:10.953096 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96a2f0-1b8c-4f47-a690-00355db5b701 service cinder] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:11.081059 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:11.180034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5477dc-9c82-466d-b97c-5b79ffd4aa6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:11.181108 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5477dc-9c82-466d-b97c-5b79ffd4aa6f service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:11.181819 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5477dc-9c82-466d-b97c-5b79ffd4aa6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:11.190897 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5477dc-9c82-466d-b97c-5b79ffd4aa6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:11.198878 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-0f5477dc-9c82-466d-b97c-5b79ffd4aa6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:11.226441 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd1231-7723-44ab-8006-b9a0b057aa24 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:11.255830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78242e0-dc30-44d3-b329-61b33e523f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:11.258700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78242e0-dc30-44d3-b329-61b33e523f35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:11.260499 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78242e0-dc30-44d3-b329-61b33e523f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:11.260798 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78242e0-dc30-44d3-b329-61b33e523f35 admin admin] PATH_INFO: `/v3/projects/6f3d02f6185e4172affd30b50895f9c1/users/d3784a3d9d224263a977da405cbb789d/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:11.291036 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5477dc-9c82-466d-b97c-5b79ffd4aa6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:11.292876 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5477dc-9c82-466d-b97c-5b79ffd4aa6f service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:11.318656 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78242e0-dc30-44d3-b329-61b33e523f35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f3d02f6185e4172affd30b50895f9c1, user_id=d3784a3d9d224263a977da405cbb789d, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:11.325110 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78242e0-dc30-44d3-b329-61b33e523f35 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:11.352093 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 888/1768] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 730 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:11.370844 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 881/1769] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:10 2022] PUT /identity/v3/projects/6f3d02f6185e4172affd30b50895f9c1/users/d3784a3d9d224263a977da405cbb789d/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:11.384366 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee95c34-181b-4055-92ab-5c5bc5c6a889 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:11.384659 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee95c34-181b-4055-92ab-5c5bc5c6a889 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:11.384949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee95c34-181b-4055-92ab-5c5bc5c6a889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:11.455343 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-1ee95c34-181b-4055-92ab-5c5bc5c6a889 None None] MFA Rules not processed for user `d3784a3d9d224263a977da405cbb789d`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:11.456855 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-1ee95c34-181b-4055-92ab-5c5bc5c6a889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:11.657419 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 889/1770] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:11.667811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78242e0-dc30-44d3-b329-61b33e523f35 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:11.911313 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78242e0-dc30-44d3-b329-61b33e523f35 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:11.925626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de63a6-89ed-41d5-907f-23824f4781c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:11.925626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de63a6-89ed-41d5-907f-23824f4781c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:11.925626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de63a6-89ed-41d5-907f-23824f4781c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:11.925626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de63a6-89ed-41d5-907f-23824f4781c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:11.926969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de63a6-89ed-41d5-907f-23824f4781c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:11.928087 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de63a6-89ed-41d5-907f-23824f4781c8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:11.933751 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:12.010618 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 882/1771] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:11 2022] POST /identity/v3/projects => generated 383 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:12.016265 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee95c34-181b-4055-92ab-5c5bc5c6a889 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:12.074230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee95c34-181b-4055-92ab-5c5bc5c6a889 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:12.155907 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de63a6-89ed-41d5-907f-23824f4781c8 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:12.158135 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-28de63a6-89ed-41d5-907f-23824f4781c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:12.393913 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c06d44-e667-40cc-8f8d-0d5344234802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:12.394653 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c06d44-e667-40cc-8f8d-0d5344234802 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:12.395144 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c06d44-e667-40cc-8f8d-0d5344234802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:12.398041 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c06d44-e667-40cc-8f8d-0d5344234802 admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:12.400568 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c06d44-e667-40cc-8f8d-0d5344234802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:12.406952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c06d44-e667-40cc-8f8d-0d5344234802 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:12.461332 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de63a6-89ed-41d5-907f-23824f4781c8 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:12.477294 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeafe2d2-0b93-4ff9-ae93-4883e56ee259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:12.479577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafe2d2-0b93-4ff9-ae93-4883e56ee259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:12.480971 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafe2d2-0b93-4ff9-ae93-4883e56ee259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:12.482247 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 890/1772] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:12 2022] POST /identity/v3/users => generated 298 bytes in 468 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:12.482724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafe2d2-0b93-4ff9-ae93-4883e56ee259 admin admin] PATH_INFO: `/v3/projects/810abdc029ae4b71b375590bbd1a8ceb` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:12.488247 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c06d44-e667-40cc-8f8d-0d5344234802 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:12.492167 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeafe2d2-0b93-4ff9-ae93-4883e56ee259 admin admin] RBAC: Authorizing `identity:delete_project(project_id=810abdc029ae4b71b375590bbd1a8ceb)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:12.505801 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeafe2d2-0b93-4ff9-ae93-4883e56ee259 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:12.517796 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c06d44-e667-40cc-8f8d-0d5344234802 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:12.525687 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523121b-058f-4935-bdd1-09e61a5755ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:12.526292 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523121b-058f-4935-bdd1-09e61a5755ab admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:12.526823 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523121b-058f-4935-bdd1-09e61a5755ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:12.527219 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523121b-058f-4935-bdd1-09e61a5755ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:12.529144 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1523121b-058f-4935-bdd1-09e61a5755ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:12.532281 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1523121b-058f-4935-bdd1-09e61a5755ab admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:12.563548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 891/1773] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:12.568701 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523121b-058f-4935-bdd1-09e61a5755ab admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:12.610713 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523121b-058f-4935-bdd1-09e61a5755ab admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:12.645164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed731b-ca9d-4682-92b9-f7fedf8ea000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:12.650231 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed731b-ca9d-4682-92b9-f7fedf8ea000 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:12.651182 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed731b-ca9d-4682-92b9-f7fedf8ea000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:12.658596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed731b-ca9d-4682-92b9-f7fedf8ea000 admin admin] PATH_INFO: `/v3/projects/8aca7a3f3d154c87ad611ce8105aa50f/users/e9557ba84c57407f86ae74f95a960797/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:12.718886 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed731b-ca9d-4682-92b9-f7fedf8ea000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aca7a3f3d154c87ad611ce8105aa50f, user_id=e9557ba84c57407f86ae74f95a960797, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:12.723282 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed731b-ca9d-4682-92b9-f7fedf8ea000 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:12.727438 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-eeafe2d2-0b93-4ff9-ae93-4883e56ee259 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '810abdc029ae4b71b375590bbd1a8ceb', 'request_id': 'req-eeafe2d2-0b93-4ff9-ae93-4883e56ee259'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:12.782798 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 892/1774] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:12 2022] PUT /identity/v3/projects/8aca7a3f3d154c87ad611ce8105aa50f/users/e9557ba84c57407f86ae74f95a960797/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:12.785470 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-eeafe2d2-0b93-4ff9-ae93-4883e56ee259 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '810abdc029ae4b71b375590bbd1a8ceb', 'request_id': 'req-eeafe2d2-0b93-4ff9-ae93-4883e56ee259'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:12.788878 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed731b-ca9d-4682-92b9-f7fedf8ea000 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:12.833047 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-eeafe2d2-0b93-4ff9-ae93-4883e56ee259 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '810abdc029ae4b71b375590bbd1a8ceb', 'request_id': 'req-eeafe2d2-0b93-4ff9-ae93-4883e56ee259'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:12.891834 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:12.917375 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-eeafe2d2-0b93-4ff9-ae93-4883e56ee259 admin admin] The token cache is being invalidate because project 810abdc029ae4b71b375590bbd1a8ceb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:12.918722 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-eeafe2d2-0b93-4ff9-ae93-4883e56ee259 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:12.931004 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-eeafe2d2-0b93-4ff9-ae93-4883e56ee259 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:12.934601 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 883/1775] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:12 2022] DELETE /identity/v3/projects/810abdc029ae4b71b375590bbd1a8ceb => generated 0 bytes in 779 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:12:12.938904 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7a0a4-66fc-404c-9fc1-6b3c520dedc4 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:12.939701 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7a0a4-66fc-404c-9fc1-6b3c520dedc4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:12.940379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7a0a4-66fc-404c-9fc1-6b3c520dedc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:13.019212 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-43f7a0a4-66fc-404c-9fc1-6b3c520dedc4 None None] MFA Rules not processed for user `5a84f8aed50c4b3e8e5b33c5b587b26e`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:13.021187 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-43f7a0a4-66fc-404c-9fc1-6b3c520dedc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:13.104993 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed731b-ca9d-4682-92b9-f7fedf8ea000 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:13.139373 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c30c0a-b2e3-468b-9ba2-8da98c844b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:13.140670 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c30c0a-b2e3-468b-9ba2-8da98c844b37 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:13.141154 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c30c0a-b2e3-468b-9ba2-8da98c844b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:13.141635 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c30c0a-b2e3-468b-9ba2-8da98c844b37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:13.142506 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c30c0a-b2e3-468b-9ba2-8da98c844b37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:13.164590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c30c0a-b2e3-468b-9ba2-8da98c844b37 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:13.239482 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 893/1776] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 453 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:13.241490 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e6463-b2ee-40ea-bd9f-909d77d84632 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:13.241827 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e6463-b2ee-40ea-bd9f-909d77d84632 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:13.242118 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e6463-b2ee-40ea-bd9f-909d77d84632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:13.319919 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 884/1777] 10.222.0.25 () {58 vars in 1015 bytes} [Thu Jun 16 16:12:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:13.327200 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7a0a4-66fc-404c-9fc1-6b3c520dedc4 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:13.331752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-43f7a0a4-66fc-404c-9fc1-6b3c520dedc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:13.422850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7a0a4-66fc-404c-9fc1-6b3c520dedc4 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:13.424968 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-212e6463-b2ee-40ea-bd9f-909d77d84632 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:13.430161 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-212e6463-b2ee-40ea-bd9f-909d77d84632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:13.451320 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64eb3e-5982-4398-9b68-dbdddf6c8c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:13.454820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64eb3e-5982-4398-9b68-dbdddf6c8c4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:13.461662 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64eb3e-5982-4398-9b68-dbdddf6c8c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:13.462385 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64eb3e-5982-4398-9b68-dbdddf6c8c4b admin admin] PATH_INFO: `/v3/projects/8aca7a3f3d154c87ad611ce8105aa50f/users/e9557ba84c57407f86ae74f95a960797/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:13.480617 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64eb3e-5982-4398-9b68-dbdddf6c8c4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aca7a3f3d154c87ad611ce8105aa50f, user_id=e9557ba84c57407f86ae74f95a960797, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:13.495965 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 894/1778] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:13.500529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e6463-b2ee-40ea-bd9f-909d77d84632 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:13.506275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64eb3e-5982-4398-9b68-dbdddf6c8c4b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:13.517088 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-212e6463-b2ee-40ea-bd9f-909d77d84632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:13.527695 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 885/1779] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:13 2022] PUT /identity/v3/projects/8aca7a3f3d154c87ad611ce8105aa50f/users/e9557ba84c57407f86ae74f95a960797/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:13.530300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e595d-6064-4968-bc49-00cd991522cb None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:13.530904 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e595d-6064-4968-bc49-00cd991522cb None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:13.531550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e595d-6064-4968-bc49-00cd991522cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:13.568718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e6463-b2ee-40ea-bd9f-909d77d84632 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:13.615277 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-403e595d-6064-4968-bc49-00cd991522cb None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:13.621579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-403e595d-6064-4968-bc49-00cd991522cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:13.652994 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:13.790314 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb91876-a600-4183-8685-36c1514f9ac6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:13.791175 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb91876-a600-4183-8685-36c1514f9ac6 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:13.791465 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb91876-a600-4183-8685-36c1514f9ac6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:13.791753 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb91876-a600-4183-8685-36c1514f9ac6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:13.823542 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb91876-a600-4183-8685-36c1514f9ac6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:13.834220 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb91876-a600-4183-8685-36c1514f9ac6 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:13.848181 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 895/1780] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:13.849490 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9cba4c-1edb-47c5-90a4-d4bff6d1a36e None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:13.849780 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9cba4c-1edb-47c5-90a4-d4bff6d1a36e None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:13.850109 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9cba4c-1edb-47c5-90a4-d4bff6d1a36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:13.940980 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 886/1781] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:13.943052 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e595d-6064-4968-bc49-00cd991522cb None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:13.949868 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-403e595d-6064-4968-bc49-00cd991522cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:13.984401 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e595d-6064-4968-bc49-00cd991522cb None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:13.991441 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-2e9cba4c-1edb-47c5-90a4-d4bff6d1a36e None None] MFA Rules not processed for user `e9557ba84c57407f86ae74f95a960797`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:13.992965 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-2e9cba4c-1edb-47c5-90a4-d4bff6d1a36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:14.012648 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ffefc-3766-416a-9a8b-74861adcf93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:14.013587 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ffefc-3766-416a-9a8b-74861adcf93a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:14.013871 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ffefc-3766-416a-9a8b-74861adcf93a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:14.014180 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ffefc-3766-416a-9a8b-74861adcf93a admin admin] PATH_INFO: `/v3/projects/830fb8d7d68d4e49af22dbbeec2e8146` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:14.018332 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ffefc-3766-416a-9a8b-74861adcf93a admin admin] RBAC: Authorizing `identity:delete_project(project_id=830fb8d7d68d4e49af22dbbeec2e8146)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:14.020486 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ffefc-3766-416a-9a8b-74861adcf93a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:14.233862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-964ffefc-3766-416a-9a8b-74861adcf93a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '830fb8d7d68d4e49af22dbbeec2e8146', 'request_id': 'req-964ffefc-3766-416a-9a8b-74861adcf93a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:14.241181 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 896/1782] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:14.243112 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9cba4c-1edb-47c5-90a4-d4bff6d1a36e None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:14.260334 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-964ffefc-3766-416a-9a8b-74861adcf93a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '830fb8d7d68d4e49af22dbbeec2e8146', 'request_id': 'req-964ffefc-3766-416a-9a8b-74861adcf93a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:14.286641 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-964ffefc-3766-416a-9a8b-74861adcf93a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '830fb8d7d68d4e49af22dbbeec2e8146', 'request_id': 'req-964ffefc-3766-416a-9a8b-74861adcf93a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:14.311832 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9cba4c-1edb-47c5-90a4-d4bff6d1a36e None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:14.335688 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-964ffefc-3766-416a-9a8b-74861adcf93a admin admin] The token cache is being invalidate because project 830fb8d7d68d4e49af22dbbeec2e8146 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:14.335688 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-964ffefc-3766-416a-9a8b-74861adcf93a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:14.337638 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-964ffefc-3766-416a-9a8b-74861adcf93a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:14.348342 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 887/1783] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:13 2022] DELETE /identity/v3/projects/830fb8d7d68d4e49af22dbbeec2e8146 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:14.353821 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ffefc-3766-416a-9a8b-74861adcf93a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:14.359007 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-964ffefc-3766-416a-9a8b-74861adcf93a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:14.439810 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:14.536909 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266f461-b085-48cb-81c4-46819af1b71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:14.537801 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266f461-b085-48cb-81c4-46819af1b71c admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:14.538276 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266f461-b085-48cb-81c4-46819af1b71c admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:14.538671 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266f461-b085-48cb-81c4-46819af1b71c admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:14.539586 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266f461-b085-48cb-81c4-46819af1b71c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:12:14.542175 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266f461-b085-48cb-81c4-46819af1b71c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:14.544096 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266f461-b085-48cb-81c4-46819af1b71c admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:14.589262 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 897/1784] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:12:14.589262 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266f461-b085-48cb-81c4-46819af1b71c admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:14.597250 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-f266f461-b085-48cb-81c4-46819af1b71c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:14.619854 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266f461-b085-48cb-81c4-46819af1b71c admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:14.632480 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8c49f-c2bd-4e40-b6b2-6b6974dfddd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:14.633727 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8c49f-c2bd-4e40-b6b2-6b6974dfddd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:14.634109 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8c49f-c2bd-4e40-b6b2-6b6974dfddd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:14.634460 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8c49f-c2bd-4e40-b6b2-6b6974dfddd0 admin admin] PATH_INFO: `/v3/users/73a0706e235346c8a860cb0c0f07f027` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:14.638762 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8c49f-c2bd-4e40-b6b2-6b6974dfddd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73a0706e235346c8a860cb0c0f07f027)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:14.641065 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8c49f-c2bd-4e40-b6b2-6b6974dfddd0 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:14.688315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ffefc-3766-416a-9a8b-74861adcf93a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:14.702326 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a993904-5675-49f7-949e-47971a0e692e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:14.703131 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a993904-5675-49f7-949e-47971a0e692e service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:14.703422 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a993904-5675-49f7-949e-47971a0e692e service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:14.703958 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a993904-5675-49f7-949e-47971a0e692e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:14.712247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7a993904-5675-49f7-949e-47971a0e692e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:14.890109 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:14.942128 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a993904-5675-49f7-949e-47971a0e692e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:14.950297 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a993904-5675-49f7-949e-47971a0e692e service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:14.954837 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-c6e8c49f-c2bd-4e40-b6b2-6b6974dfddd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73a0706e235346c8a860cb0c0f07f027'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:14.995433 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 888/1785] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:14.998343 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a993904-5675-49f7-949e-47971a0e692e service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:15.005416 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7a993904-5675-49f7-949e-47971a0e692e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:15.021819 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-c6e8c49f-c2bd-4e40-b6b2-6b6974dfddd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73a0706e235346c8a860cb0c0f07f027'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:15.072594 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a993904-5675-49f7-949e-47971a0e692e service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:15.103817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-c6e8c49f-c2bd-4e40-b6b2-6b6974dfddd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73a0706e235346c8a860cb0c0f07f027'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:15.108052 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9322637-6255-40bf-a6c6-561ebe19b9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:15.108963 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9322637-6255-40bf-a6c6-561ebe19b9da admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:15.109051 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9322637-6255-40bf-a6c6-561ebe19b9da admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:15.109379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9322637-6255-40bf-a6c6-561ebe19b9da admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:15.112676 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-c6e8c49f-c2bd-4e40-b6b2-6b6974dfddd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73a0706e235346c8a860cb0c0f07f027'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:15.119198 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9322637-6255-40bf-a6c6-561ebe19b9da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:15.132839 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9322637-6255-40bf-a6c6-561ebe19b9da admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:15.156514 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-c6e8c49f-c2bd-4e40-b6b2-6b6974dfddd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73a0706e235346c8a860cb0c0f07f027'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:15.206427 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 898/1786] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:14 2022] DELETE /identity/v3/users/73a0706e235346c8a860cb0c0f07f027 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:15.286741 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 889/1787] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:14 2022] POST /identity/v3/projects => generated 375 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:15.291702 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8c49f-c2bd-4e40-b6b2-6b6974dfddd0 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:15.294982 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-c6e8c49f-c2bd-4e40-b6b2-6b6974dfddd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:15.303519 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9322637-6255-40bf-a6c6-561ebe19b9da admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:15.310982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d9322637-6255-40bf-a6c6-561ebe19b9da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:15.497991 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8c49f-c2bd-4e40-b6b2-6b6974dfddd0 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:15.625148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:15.661724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9322637-6255-40bf-a6c6-561ebe19b9da admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:15.675194 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e919b575-0cc0-448a-ba82-97b0b3c4de16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:15.681743 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919b575-0cc0-448a-ba82-97b0b3c4de16 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:15.682294 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919b575-0cc0-448a-ba82-97b0b3c4de16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:15.682605 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919b575-0cc0-448a-ba82-97b0b3c4de16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:15.689134 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e919b575-0cc0-448a-ba82-97b0b3c4de16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:15.694458 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b788b-567b-4a40-886a-da619213afa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:15.695759 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b788b-567b-4a40-886a-da619213afa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:15.695759 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b788b-567b-4a40-886a-da619213afa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:15.698241 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b788b-567b-4a40-886a-da619213afa3 admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:15.699401 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1b788b-567b-4a40-886a-da619213afa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:15.705193 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1b788b-567b-4a40-886a-da619213afa3 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:15.812407 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 899/1788] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:15 2022] POST /identity/v3/users => generated 288 bytes in 518 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:15.815275 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b788b-567b-4a40-886a-da619213afa3 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:15.845531 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b788b-567b-4a40-886a-da619213afa3 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:15.860643 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd373b1-d94d-443d-ad8d-799ee33f0a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:15.862081 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd373b1-d94d-443d-ad8d-799ee33f0a9e admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:15.862081 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd373b1-d94d-443d-ad8d-799ee33f0a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:15.862290 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd373b1-d94d-443d-ad8d-799ee33f0a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:15.863249 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd373b1-d94d-443d-ad8d-799ee33f0a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:15.865424 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd373b1-d94d-443d-ad8d-799ee33f0a9e admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:15.915759 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 900/1789] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:15.918845 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd373b1-d94d-443d-ad8d-799ee33f0a9e admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:15.960023 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd373b1-d94d-443d-ad8d-799ee33f0a9e admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:15.982977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16493601-3771-4b45-988f-501f95c54c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:15.983675 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16493601-3771-4b45-988f-501f95c54c7d admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:15.983961 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16493601-3771-4b45-988f-501f95c54c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:15.984242 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16493601-3771-4b45-988f-501f95c54c7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:15.985057 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16493601-3771-4b45-988f-501f95c54c7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:15.986740 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e919b575-0cc0-448a-ba82-97b0b3c4de16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:15.991230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16493601-3771-4b45-988f-501f95c54c7d admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:15.992248 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e919b575-0cc0-448a-ba82-97b0b3c4de16 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:16.032798 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 890/1790] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 731 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:16.033789 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 901/1791] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:16.034984 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e919b575-0cc0-448a-ba82-97b0b3c4de16 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:16.087311 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e919b575-0cc0-448a-ba82-97b0b3c4de16 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:16.104330 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d852e-f2b7-4571-ba4e-69a9672d8f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:16.106215 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d852e-f2b7-4571-ba4e-69a9672d8f0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:16.107291 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d852e-f2b7-4571-ba4e-69a9672d8f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:16.107812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d852e-f2b7-4571-ba4e-69a9672d8f0a admin admin] PATH_INFO: `/v3/projects/4a1c4db251e347f79bc75af8eef192c1/users/e792b600ea114bb0bc76bda235016377/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:16.208862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d852e-f2b7-4571-ba4e-69a9672d8f0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a1c4db251e347f79bc75af8eef192c1, user_id=e792b600ea114bb0bc76bda235016377, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:16.225969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d852e-f2b7-4571-ba4e-69a9672d8f0a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:16.284620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 891/1792] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:16 2022] PUT /identity/v3/projects/4a1c4db251e347f79bc75af8eef192c1/users/e792b600ea114bb0bc76bda235016377/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:16.297553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203513f1-289c-4558-85c9-06c6a1ce9037 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:16.298107 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203513f1-289c-4558-85c9-06c6a1ce9037 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:16.298357 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203513f1-289c-4558-85c9-06c6a1ce9037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:16.431512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-203513f1-289c-4558-85c9-06c6a1ce9037 None None] MFA Rules not processed for user `e792b600ea114bb0bc76bda235016377`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:16.438527 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-203513f1-289c-4558-85c9-06c6a1ce9037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:16.799184 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 902/1793] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 503 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:16.802849 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d852e-f2b7-4571-ba4e-69a9672d8f0a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:16.884665 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203513f1-289c-4558-85c9-06c6a1ce9037 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:16.888001 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-203513f1-289c-4558-85c9-06c6a1ce9037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:17.108107 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:17.166235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d852e-f2b7-4571-ba4e-69a9672d8f0a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:17.210346 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ae5ae-97e9-4a2a-bd84-3e662a09eefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:17.211228 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ae5ae-97e9-4a2a-bd84-3e662a09eefe admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:17.213233 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ae5ae-97e9-4a2a-bd84-3e662a09eefe admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:17.213654 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ae5ae-97e9-4a2a-bd84-3e662a09eefe admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:17.215308 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ae5ae-97e9-4a2a-bd84-3e662a09eefe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:17.223140 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ae5ae-97e9-4a2a-bd84-3e662a09eefe admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:17.234550 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203513f1-289c-4558-85c9-06c6a1ce9037 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:17.263498 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2538e-ff9c-487d-972d-b7844010ff16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:17.265074 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2538e-ff9c-487d-972d-b7844010ff16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:17.265716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2538e-ff9c-487d-972d-b7844010ff16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:17.266260 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2538e-ff9c-487d-972d-b7844010ff16 admin admin] PATH_INFO: `/v3/users/d3784a3d9d224263a977da405cbb789d` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:17.275099 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad2538e-ff9c-487d-972d-b7844010ff16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3784a3d9d224263a977da405cbb789d)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:17.300528 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad2538e-ff9c-487d-972d-b7844010ff16 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:17.386972 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 892/1794] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:16 2022] POST /identity/v3/projects => generated 375 bytes in 586 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:17.397536 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ae5ae-97e9-4a2a-bd84-3e662a09eefe admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:17.553905 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-1ad2538e-ff9c-487d-972d-b7844010ff16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3784a3d9d224263a977da405cbb789d'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:17.555921 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:17.597023 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-1ad2538e-ff9c-487d-972d-b7844010ff16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3784a3d9d224263a977da405cbb789d'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:17.631395 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ae5ae-97e9-4a2a-bd84-3e662a09eefe admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:17.634552 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-1ad2538e-ff9c-487d-972d-b7844010ff16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3784a3d9d224263a977da405cbb789d'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:17.636365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-1ad2538e-ff9c-487d-972d-b7844010ff16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3784a3d9d224263a977da405cbb789d'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:17.650580 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9b589-bae1-4e9d-9dcf-50a1440faffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:17.654178 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9b589-bae1-4e9d-9dcf-50a1440faffe admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:17.657030 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-1ad2538e-ff9c-487d-972d-b7844010ff16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3784a3d9d224263a977da405cbb789d'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:17.657726 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9b589-bae1-4e9d-9dcf-50a1440faffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:17.661912 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9b589-bae1-4e9d-9dcf-50a1440faffe admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:17.663670 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc9b589-bae1-4e9d-9dcf-50a1440faffe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:17.670576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc9b589-bae1-4e9d-9dcf-50a1440faffe admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:17.674333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 903/1795] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:16 2022] DELETE /identity/v3/users/d3784a3d9d224263a977da405cbb789d => generated 0 bytes in 790 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:17.679621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420dc33-1618-426d-8639-3ea5de387108 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:17.680046 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420dc33-1618-426d-8639-3ea5de387108 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:17.680697 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420dc33-1618-426d-8639-3ea5de387108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:17.730637 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 893/1796] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:17 2022] POST /identity/v3/users => generated 294 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:17.734276 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9b589-bae1-4e9d-9dcf-50a1440faffe admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:17.743786 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9cc9b589-bae1-4e9d-9dcf-50a1440faffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:17.780977 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9b589-bae1-4e9d-9dcf-50a1440faffe admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:17.812468 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-c420dc33-1618-426d-8639-3ea5de387108 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:17.827382 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-c420dc33-1618-426d-8639-3ea5de387108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:18.047430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 904/1797] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:18.193699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b4dda-c050-4440-ae41-f347b3b93be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:18.194738 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b4dda-c050-4440-ae41-f347b3b93be8 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:18.202403 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b4dda-c050-4440-ae41-f347b3b93be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:18.204679 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b4dda-c050-4440-ae41-f347b3b93be8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:18.205466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b4dda-c050-4440-ae41-f347b3b93be8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:18.229411 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b4dda-c050-4440-ae41-f347b3b93be8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:18.264009 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420dc33-1618-426d-8639-3ea5de387108 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:18.269402 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-c420dc33-1618-426d-8639-3ea5de387108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:18.282170 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 894/1798] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 550 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:12:18.286409 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b4dda-c050-4440-ae41-f347b3b93be8 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:18.318701 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b4dda-c050-4440-ae41-f347b3b93be8 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:18.360856 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02d43f-956c-4448-b7ca-73d68aacc1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:18.363225 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02d43f-956c-4448-b7ca-73d68aacc1b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:18.363895 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02d43f-956c-4448-b7ca-73d68aacc1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:18.364192 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02d43f-956c-4448-b7ca-73d68aacc1b0 admin admin] PATH_INFO: `/v3/projects/d904faabfae04291b048160634586b66/users/8f71730d94dc47b68154b4d6f6f51ba1/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:18.454490 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b02d43f-956c-4448-b7ca-73d68aacc1b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d904faabfae04291b048160634586b66, user_id=8f71730d94dc47b68154b4d6f6f51ba1, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:18.463226 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b02d43f-956c-4448-b7ca-73d68aacc1b0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:18.514525 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 895/1799] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:18 2022] PUT /identity/v3/projects/d904faabfae04291b048160634586b66/users/8f71730d94dc47b68154b4d6f6f51ba1/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:18.516514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0863cfe-7218-493e-91ee-177af589c508 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:18.516807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0863cfe-7218-493e-91ee-177af589c508 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:18.517083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0863cfe-7218-493e-91ee-177af589c508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:18.567382 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420dc33-1618-426d-8639-3ea5de387108 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:18.593766 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f0863cfe-7218-493e-91ee-177af589c508 None None] MFA Rules not processed for user `80386d7afc994931af05fd49fa8043d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:18.596655 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f0863cfe-7218-493e-91ee-177af589c508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:18.729453 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e34ef-3d2f-4790-a692-ca37552fb44f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:18.730713 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e34ef-3d2f-4790-a692-ca37552fb44f service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:18.731254 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e34ef-3d2f-4790-a692-ca37552fb44f service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:18.732701 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e34ef-3d2f-4790-a692-ca37552fb44f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:18.777688 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:18.790456 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:18.818466 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e34ef-3d2f-4790-a692-ca37552fb44f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:18.823682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e34ef-3d2f-4790-a692-ca37552fb44f service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:18.838442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 896/1800] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:18.839588 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0863cfe-7218-493e-91ee-177af589c508 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:18.842540 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 905/1801] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:18.855549 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0863cfe-7218-493e-91ee-177af589c508 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:18.870173 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417795c-db99-4152-b408-6c7f93925939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:18.871187 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417795c-db99-4152-b408-6c7f93925939 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:18.871696 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417795c-db99-4152-b408-6c7f93925939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:18.872936 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417795c-db99-4152-b408-6c7f93925939 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:18.873854 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9417795c-db99-4152-b408-6c7f93925939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:18.875724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9417795c-db99-4152-b408-6c7f93925939 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:18.906305 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 897/1802] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:18.910417 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e34ef-3d2f-4790-a692-ca37552fb44f service neutron] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:18.927409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e34ef-3d2f-4790-a692-ca37552fb44f service neutron] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:18.940430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81264605-65ce-4c15-8148-57a3930f38c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:18.943532 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81264605-65ce-4c15-8148-57a3930f38c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:18.943954 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81264605-65ce-4c15-8148-57a3930f38c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:18.944850 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81264605-65ce-4c15-8148-57a3930f38c9 admin admin] PATH_INFO: `/v3/projects/d904faabfae04291b048160634586b66/users/8f71730d94dc47b68154b4d6f6f51ba1/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:18.952118 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81264605-65ce-4c15-8148-57a3930f38c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d904faabfae04291b048160634586b66, user_id=8f71730d94dc47b68154b4d6f6f51ba1, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:18.959409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81264605-65ce-4c15-8148-57a3930f38c9 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:18.988475 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 906/1803] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:18 2022] PUT /identity/v3/projects/d904faabfae04291b048160634586b66/users/8f71730d94dc47b68154b4d6f6f51ba1/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:19.003813 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea2471-a3af-4c88-b358-f0602bf63e61 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:19.004469 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea2471-a3af-4c88-b358-f0602bf63e61 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:19.004889 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea2471-a3af-4c88-b358-f0602bf63e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:19.052928 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-78ea2471-a3af-4c88-b358-f0602bf63e61 None None] MFA Rules not processed for user `8f71730d94dc47b68154b4d6f6f51ba1`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:19.054683 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-78ea2471-a3af-4c88-b358-f0602bf63e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:19.062534 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c270a69-a12e-4e26-8b87-c5605c65cf88 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:19.063288 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c270a69-a12e-4e26-8b87-c5605c65cf88 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:19.063947 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c270a69-a12e-4e26-8b87-c5605c65cf88 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:19.065803 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 907/1804] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jun 16 16:12:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:12:19.086298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee569267-67d5-47eb-a6c7-301377d15800 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:19.086941 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee569267-67d5-47eb-a6c7-301377d15800 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:19.087443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee569267-67d5-47eb-a6c7-301377d15800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:19.126653 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-ee569267-67d5-47eb-a6c7-301377d15800 None None] MFA Rules not processed for user `c3645b1a0be34c01ac87c734fcd85ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:19.130152 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-ee569267-67d5-47eb-a6c7-301377d15800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:19.319093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 898/1805] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:19 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:19.329363 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 908/1806] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jun 16 16:12:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:12:19.333574 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea2471-a3af-4c88-b358-f0602bf63e61 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:19.338193 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-78ea2471-a3af-4c88-b358-f0602bf63e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:19.386723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee569267-67d5-47eb-a6c7-301377d15800 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:19.389096 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-ee569267-67d5-47eb-a6c7-301377d15800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:19.565837 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee569267-67d5-47eb-a6c7-301377d15800 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:19.586998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0fefe-9dda-42b3-bdd7-52888c14768d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:19.587924 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0fefe-9dda-42b3-bdd7-52888c14768d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:19.588221 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0fefe-9dda-42b3-bdd7-52888c14768d admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:19.588479 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0fefe-9dda-42b3-bdd7-52888c14768d admin admin] PATH_INFO: `/v3/projects/51ddf36d52354a0b94334f62d5cea431` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:19.596410 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d0fefe-9dda-42b3-bdd7-52888c14768d admin admin] RBAC: Authorizing `identity:delete_project(project_id=51ddf36d52354a0b94334f62d5cea431)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:19.602394 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea2471-a3af-4c88-b358-f0602bf63e61 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:19.604927 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d0fefe-9dda-42b3-bdd7-52888c14768d admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:19.615666 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3541db-7839-4ac5-a00a-78fa80f0b356 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:19.616369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3541db-7839-4ac5-a00a-78fa80f0b356 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:19.616692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3541db-7839-4ac5-a00a-78fa80f0b356 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:19.626184 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3541db-7839-4ac5-a00a-78fa80f0b356 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:19.633008 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3b3541db-7839-4ac5-a00a-78fa80f0b356 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:19.771575 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-32d0fefe-9dda-42b3-bdd7-52888c14768d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51ddf36d52354a0b94334f62d5cea431', 'request_id': 'req-32d0fefe-9dda-42b3-bdd7-52888c14768d'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:19.777498 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:19.814443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-32d0fefe-9dda-42b3-bdd7-52888c14768d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51ddf36d52354a0b94334f62d5cea431', 'request_id': 'req-32d0fefe-9dda-42b3-bdd7-52888c14768d'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:19.825021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-32d0fefe-9dda-42b3-bdd7-52888c14768d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51ddf36d52354a0b94334f62d5cea431', 'request_id': 'req-32d0fefe-9dda-42b3-bdd7-52888c14768d'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:19.847866 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3541db-7839-4ac5-a00a-78fa80f0b356 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:19.848792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-32d0fefe-9dda-42b3-bdd7-52888c14768d admin admin] The token cache is being invalidate because project 51ddf36d52354a0b94334f62d5cea431 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:19.849483 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3541db-7839-4ac5-a00a-78fa80f0b356 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:19.849677 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-32d0fefe-9dda-42b3-bdd7-52888c14768d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:19.851110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-32d0fefe-9dda-42b3-bdd7-52888c14768d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:19.854231 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 909/1807] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:19 2022] DELETE /identity/v3/projects/51ddf36d52354a0b94334f62d5cea431 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:12:19.862993 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cdbc7-4a68-4b38-8d67-c3a957269621 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:19.863294 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cdbc7-4a68-4b38-8d67-c3a957269621 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:19.863574 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cdbc7-4a68-4b38-8d67-c3a957269621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:19.890068 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 899/1808] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:19.898398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-0b7cdbc7-4a68-4b38-8d67-c3a957269621 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:19.900639 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-0b7cdbc7-4a68-4b38-8d67-c3a957269621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:20.022664 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 910/1809] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:20.023798 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c33ab-0391-4e75-b82f-89edc554e05b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.024094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c33ab-0391-4e75-b82f-89edc554e05b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.024379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c33ab-0391-4e75-b82f-89edc554e05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.085543 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-382c33ab-0391-4e75-b82f-89edc554e05b None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:20.087018 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-382c33ab-0391-4e75-b82f-89edc554e05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:20.103111 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 900/1810] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:20.109528 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cdbc7-4a68-4b38-8d67-c3a957269621 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:20.128360 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cdbc7-4a68-4b38-8d67-c3a957269621 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:20.144527 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f6d2c-e747-49fc-840d-beeff1b50d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:20.147166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f6d2c-e747-49fc-840d-beeff1b50d76 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.147506 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f6d2c-e747-49fc-840d-beeff1b50d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.149570 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f6d2c-e747-49fc-840d-beeff1b50d76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.153070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f6d2c-e747-49fc-840d-beeff1b50d76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:12:20.156319 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f6d2c-e747-49fc-840d-beeff1b50d76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:20.158985 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f6d2c-e747-49fc-840d-beeff1b50d76 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:20.193150 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 911/1811] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:12:20.194014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c33ab-0391-4e75-b82f-89edc554e05b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:20.205518 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c33ab-0391-4e75-b82f-89edc554e05b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:20.212001 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa0ff1-95be-4c8c-a889-834c3d1a6a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:20.251868 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa0ff1-95be-4c8c-a889-834c3d1a6a03 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.253684 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa0ff1-95be-4c8c-a889-834c3d1a6a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.255011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa0ff1-95be-4c8c-a889-834c3d1a6a03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.258233 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa0ff1-95be-4c8c-a889-834c3d1a6a03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:20.264584 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa0ff1-95be-4c8c-a889-834c3d1a6a03 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:20.273221 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:20.411601 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 901/1812] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:20 2022] POST /identity/v3/projects => generated 379 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:20.413483 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f6d2c-e747-49fc-840d-beeff1b50d76 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:20.436946 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f6d2c-e747-49fc-840d-beeff1b50d76 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:20.855729 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e616c0b-9ee7-4ef8-b27a-f6064f624ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:20.859354 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e616c0b-9ee7-4ef8-b27a-f6064f624ceb admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.859805 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e616c0b-9ee7-4ef8-b27a-f6064f624ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.862862 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e616c0b-9ee7-4ef8-b27a-f6064f624ceb admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.863705 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e616c0b-9ee7-4ef8-b27a-f6064f624ceb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:20.865739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e616c0b-9ee7-4ef8-b27a-f6064f624ceb admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:20.957311 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 912/1813] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:20 2022] POST /identity/v3/users => generated 290 bytes in 544 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:20.960382 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa0ff1-95be-4c8c-a889-834c3d1a6a03 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:20.985418 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa0ff1-95be-4c8c-a889-834c3d1a6a03 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.017772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf390f9-9f00-43ef-848c-24a57f0c40c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.018756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf390f9-9f00-43ef-848c-24a57f0c40c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.019181 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf390f9-9f00-43ef-848c-24a57f0c40c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.022029 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf390f9-9f00-43ef-848c-24a57f0c40c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.022821 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf390f9-9f00-43ef-848c-24a57f0c40c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.037073 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf390f9-9f00-43ef-848c-24a57f0c40c9 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.078446 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 902/1814] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:21.078926 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e616c0b-9ee7-4ef8-b27a-f6064f624ceb admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.108589 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e616c0b-9ee7-4ef8-b27a-f6064f624ceb admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.132179 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb928386-03f4-4ea7-9ddd-525727f3780a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.133382 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb928386-03f4-4ea7-9ddd-525727f3780a admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.135058 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb928386-03f4-4ea7-9ddd-525727f3780a admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.135457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb928386-03f4-4ea7-9ddd-525727f3780a admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.140999 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb928386-03f4-4ea7-9ddd-525727f3780a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.147907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb928386-03f4-4ea7-9ddd-525727f3780a admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.194038 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 913/1815] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:21.198427 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf390f9-9f00-43ef-848c-24a57f0c40c9 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.226571 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf390f9-9f00-43ef-848c-24a57f0c40c9 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.234333 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab6e51-5d0d-4ea1-bd88-b323c8b6dcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.236400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab6e51-5d0d-4ea1-bd88-b323c8b6dcd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.237617 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab6e51-5d0d-4ea1-bd88-b323c8b6dcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.238361 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab6e51-5d0d-4ea1-bd88-b323c8b6dcd8 admin admin] PATH_INFO: `/v3/projects/1b6cfdd94e2d43cab5b14d1027d243ed/users/d795c97cb7fc41ebab6d0cf575ff7e9a/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.270163 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab6e51-5d0d-4ea1-bd88-b323c8b6dcd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b6cfdd94e2d43cab5b14d1027d243ed, user_id=d795c97cb7fc41ebab6d0cf575ff7e9a, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.278884 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab6e51-5d0d-4ea1-bd88-b323c8b6dcd8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.309668 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 903/1816] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:21 2022] PUT /identity/v3/projects/1b6cfdd94e2d43cab5b14d1027d243ed/users/d795c97cb7fc41ebab6d0cf575ff7e9a/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:12:21.321368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5f4a9-7a09-49df-9e4c-83f011b7de5c None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.321764 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5f4a9-7a09-49df-9e4c-83f011b7de5c None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.322981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5f4a9-7a09-49df-9e4c-83f011b7de5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.407297 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-d3c5f4a9-7a09-49df-9e4c-83f011b7de5c None None] MFA Rules not processed for user `d795c97cb7fc41ebab6d0cf575ff7e9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:21.408650 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-d3c5f4a9-7a09-49df-9e4c-83f011b7de5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:21.594812 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:21.683625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db018c0c-1a54-4441-99df-3a04d7ee2269 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.684218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db018c0c-1a54-4441-99df-3a04d7ee2269 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.684218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db018c0c-1a54-4441-99df-3a04d7ee2269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.689547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 914/1817] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:21.695028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5f4a9-7a09-49df-9e4c-83f011b7de5c None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.759173 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-db018c0c-1a54-4441-99df-3a04d7ee2269 None None] MFA Rules not processed for user `2c34f05634d34da29088bcf75069958d`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:21.760537 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-db018c0c-1a54-4441-99df-3a04d7ee2269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:21.930731 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5f4a9-7a09-49df-9e4c-83f011b7de5c None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.946262 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7786b-30f4-4d5f-b280-cbd9482b795d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.947684 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7786b-30f4-4d5f-b280-cbd9482b795d admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.950260 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7786b-30f4-4d5f-b280-cbd9482b795d admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.952205 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7786b-30f4-4d5f-b280-cbd9482b795d admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.963347 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f7786b-30f4-4d5f-b280-cbd9482b795d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.971998 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 904/1818] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:21.976627 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f7786b-30f4-4d5f-b280-cbd9482b795d admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:22.049796 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 915/1819] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:21 2022] POST /identity/v3/projects => generated 383 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:12:22.053164 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db018c0c-1a54-4441-99df-3a04d7ee2269 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:22.073262 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db018c0c-1a54-4441-99df-3a04d7ee2269 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:22.238706 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a7bd8-674e-46ea-ac02-beec9bbbe3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:22.239468 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a7bd8-674e-46ea-ac02-beec9bbbe3dd admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:22.242533 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a7bd8-674e-46ea-ac02-beec9bbbe3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:22.243234 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a7bd8-674e-46ea-ac02-beec9bbbe3dd admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:22.245426 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a7bd8-674e-46ea-ac02-beec9bbbe3dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:22.249568 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a7bd8-674e-46ea-ac02-beec9bbbe3dd admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:22.289544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7786b-30f4-4d5f-b280-cbd9482b795d admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:22.293586 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-19f7786b-30f4-4d5f-b280-cbd9482b795d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:22.301407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 905/1820] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:22 2022] POST /identity/v3/users => generated 298 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:22.307111 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a7bd8-674e-46ea-ac02-beec9bbbe3dd admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:22.331502 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a7bd8-674e-46ea-ac02-beec9bbbe3dd admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:22.356705 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb375f31-037e-4612-acb8-c57fb954a63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:22.358213 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb375f31-037e-4612-acb8-c57fb954a63a admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:22.358970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb375f31-037e-4612-acb8-c57fb954a63a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:22.360018 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb375f31-037e-4612-acb8-c57fb954a63a admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:22.361608 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb375f31-037e-4612-acb8-c57fb954a63a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:22.366614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb375f31-037e-4612-acb8-c57fb954a63a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:22.403166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 906/1821] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:22.407580 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb375f31-037e-4612-acb8-c57fb954a63a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:22.448192 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb375f31-037e-4612-acb8-c57fb954a63a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:22.477896 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd843f4-f40d-4022-9f38-84bfe767e5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:22.478930 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd843f4-f40d-4022-9f38-84bfe767e5b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:22.487684 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd843f4-f40d-4022-9f38-84bfe767e5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:22.490870 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd843f4-f40d-4022-9f38-84bfe767e5b0 admin admin] PATH_INFO: `/v3/projects/c16ff4583f384dd48b6544041cff397d/users/db0cd76592b44dc18c53ed77b57be642/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:22.573237 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd843f4-f40d-4022-9f38-84bfe767e5b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c16ff4583f384dd48b6544041cff397d, user_id=db0cd76592b44dc18c53ed77b57be642, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:22.590289 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7786b-30f4-4d5f-b280-cbd9482b795d admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:22.592505 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd843f4-f40d-4022-9f38-84bfe767e5b0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:22.617165 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179708c8-be3e-4c58-91c9-4231acbf1ec0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:22.626372 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179708c8-be3e-4c58-91c9-4231acbf1ec0 service nova] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:22.627144 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179708c8-be3e-4c58-91c9-4231acbf1ec0 service nova] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:22.628148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179708c8-be3e-4c58-91c9-4231acbf1ec0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:22.631334 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-179708c8-be3e-4c58-91c9-4231acbf1ec0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:22.661886 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 907/1822] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:22 2022] PUT /identity/v3/projects/c16ff4583f384dd48b6544041cff397d/users/db0cd76592b44dc18c53ed77b57be642/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:22.662105 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179708c8-be3e-4c58-91c9-4231acbf1ec0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:22.663051 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd843f4-f40d-4022-9f38-84bfe767e5b0 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:22.664417 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179708c8-be3e-4c58-91c9-4231acbf1ec0 service nova] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:22.666735 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4fd843f4-f40d-4022-9f38-84bfe767e5b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:22.704154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:22.726161 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 916/1823] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:22.728018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179708c8-be3e-4c58-91c9-4231acbf1ec0 service nova] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:22.801101 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179708c8-be3e-4c58-91c9-4231acbf1ec0 service nova] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:22.852050 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd843f4-f40d-4022-9f38-84bfe767e5b0 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:22.866341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95896070-04a2-4153-a4df-3c10d5f8c489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:22.868221 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95896070-04a2-4153-a4df-3c10d5f8c489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:22.868626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95896070-04a2-4153-a4df-3c10d5f8c489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:22.869884 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95896070-04a2-4153-a4df-3c10d5f8c489 admin admin] PATH_INFO: `/v3/projects/6f3d02f6185e4172affd30b50895f9c1` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:22.874643 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95896070-04a2-4153-a4df-3c10d5f8c489 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f3d02f6185e4172affd30b50895f9c1)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:22.879280 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95896070-04a2-4153-a4df-3c10d5f8c489 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:23.028238 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:23.098341 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1c73a-ea45-4e74-88ac-3ad3ecc2f5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:23.099034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1c73a-ea45-4e74-88ac-3ad3ecc2f5a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:23.099318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1c73a-ea45-4e74-88ac-3ad3ecc2f5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:23.099594 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1c73a-ea45-4e74-88ac-3ad3ecc2f5a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:23.100347 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a1c73a-ea45-4e74-88ac-3ad3ecc2f5a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:23.111264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-95896070-04a2-4153-a4df-3c10d5f8c489 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f3d02f6185e4172affd30b50895f9c1', 'request_id': 'req-95896070-04a2-4153-a4df-3c10d5f8c489'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:23.119857 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a1c73a-ea45-4e74-88ac-3ad3ecc2f5a6 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:23.146722 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-95896070-04a2-4153-a4df-3c10d5f8c489 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f3d02f6185e4172affd30b50895f9c1', 'request_id': 'req-95896070-04a2-4153-a4df-3c10d5f8c489'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:23.167642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-95896070-04a2-4153-a4df-3c10d5f8c489 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f3d02f6185e4172affd30b50895f9c1', 'request_id': 'req-95896070-04a2-4153-a4df-3c10d5f8c489'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:23.174812 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 917/1824] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 448 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:23.180094 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1c73a-ea45-4e74-88ac-3ad3ecc2f5a6 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:23.235190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-95896070-04a2-4153-a4df-3c10d5f8c489 admin admin] The token cache is being invalidate because project 6f3d02f6185e4172affd30b50895f9c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:23.235490 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-95896070-04a2-4153-a4df-3c10d5f8c489 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:23.236709 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1c73a-ea45-4e74-88ac-3ad3ecc2f5a6 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:23.237789 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-95896070-04a2-4153-a4df-3c10d5f8c489 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:23.241686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 908/1825] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:22 2022] DELETE /identity/v3/projects/6f3d02f6185e4172affd30b50895f9c1 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:23.248496 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95896070-04a2-4153-a4df-3c10d5f8c489 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:23.250724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-95896070-04a2-4153-a4df-3c10d5f8c489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:23.407692 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc543056-ef32-458f-a7c5-7155c07f0a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:23.407937 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95896070-04a2-4153-a4df-3c10d5f8c489 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:23.408954 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc543056-ef32-458f-a7c5-7155c07f0a30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:23.409253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc543056-ef32-458f-a7c5-7155c07f0a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:23.410161 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc543056-ef32-458f-a7c5-7155c07f0a30 admin admin] PATH_INFO: `/v3/projects/c16ff4583f384dd48b6544041cff397d/users/db0cd76592b44dc18c53ed77b57be642/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:23.419540 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc543056-ef32-458f-a7c5-7155c07f0a30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c16ff4583f384dd48b6544041cff397d, user_id=db0cd76592b44dc18c53ed77b57be642, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:23.422898 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc543056-ef32-458f-a7c5-7155c07f0a30 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:23.437405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc36e02-35ca-422c-942b-a4eddaba569c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:23.441800 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc36e02-35ca-422c-942b-a4eddaba569c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:23.442322 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc36e02-35ca-422c-942b-a4eddaba569c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:23.442700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc36e02-35ca-422c-942b-a4eddaba569c admin admin] PATH_INFO: `/v3/users/e9557ba84c57407f86ae74f95a960797` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:23.444523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc36e02-35ca-422c-942b-a4eddaba569c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9557ba84c57407f86ae74f95a960797)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:23.447310 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc36e02-35ca-422c-942b-a4eddaba569c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:23.460317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 918/1826] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:23 2022] PUT /identity/v3/projects/c16ff4583f384dd48b6544041cff397d/users/db0cd76592b44dc18c53ed77b57be642/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:23.474477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a8127-4760-4925-9903-54334c257a20 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:23.474763 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a8127-4760-4925-9903-54334c257a20 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:23.475042 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a8127-4760-4925-9903-54334c257a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:23.539994 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:23.557378 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fdc36e02-35ca-422c-942b-a4eddaba569c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9557ba84c57407f86ae74f95a960797'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:23.595154 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-980a8127-4760-4925-9903-54334c257a20 None None] MFA Rules not processed for user `db0cd76592b44dc18c53ed77b57be642`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:23.602123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fdc36e02-35ca-422c-942b-a4eddaba569c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9557ba84c57407f86ae74f95a960797'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:23.603919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-980a8127-4760-4925-9903-54334c257a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:23.615425 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fdc36e02-35ca-422c-942b-a4eddaba569c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9557ba84c57407f86ae74f95a960797'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:23.632146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fdc36e02-35ca-422c-942b-a4eddaba569c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9557ba84c57407f86ae74f95a960797'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:23.634211 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fdc36e02-35ca-422c-942b-a4eddaba569c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9557ba84c57407f86ae74f95a960797'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:23.648708 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 909/1827] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:23 2022] DELETE /identity/v3/users/e9557ba84c57407f86ae74f95a960797 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:12:23.767726 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 919/1828] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:23.782098 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc36e02-35ca-422c-942b-a4eddaba569c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:23.784711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-fdc36e02-35ca-422c-942b-a4eddaba569c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:23.939005 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc36e02-35ca-422c-942b-a4eddaba569c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:23.945965 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4cd43-5681-4110-a1b0-777bddef5a41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:23.945965 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4cd43-5681-4110-a1b0-777bddef5a41 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:23.945965 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4cd43-5681-4110-a1b0-777bddef5a41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:23.945965 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4cd43-5681-4110-a1b0-777bddef5a41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:24.042507 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb4cd43-5681-4110-a1b0-777bddef5a41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:24.045444 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb4cd43-5681-4110-a1b0-777bddef5a41 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:24.070678 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 910/1829] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:24.212719 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a8127-4760-4925-9903-54334c257a20 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:24.301020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a8127-4760-4925-9903-54334c257a20 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:24.306536 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b34e8-9591-40fd-9272-24f984ac916f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:24.307223 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b34e8-9591-40fd-9272-24f984ac916f service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:24.307514 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b34e8-9591-40fd-9272-24f984ac916f service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:24.307797 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b34e8-9591-40fd-9272-24f984ac916f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:24.311404 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-5a1b34e8-9591-40fd-9272-24f984ac916f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:24.334851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:24.410039 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b34e8-9591-40fd-9272-24f984ac916f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:24.411278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b34e8-9591-40fd-9272-24f984ac916f service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:24.430127 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 920/1830] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:12:24.441163 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4cd43-5681-4110-a1b0-777bddef5a41 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:24.443131 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2fb4cd43-5681-4110-a1b0-777bddef5a41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:24.490159 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ce905-0722-41df-88fc-3832212273ea None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:24.490159 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ce905-0722-41df-88fc-3832212273ea None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:24.490434 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ce905-0722-41df-88fc-3832212273ea None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:24.499415 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 921/1831] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jun 16 16:12:24 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:12:24.509546 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c6454-687a-478e-91cf-390fdd709f57 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:24.510064 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c6454-687a-478e-91cf-390fdd709f57 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:24.510629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c6454-687a-478e-91cf-390fdd709f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:24.566601 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-6c1c6454-687a-478e-91cf-390fdd709f57 None None] MFA Rules not processed for user `c3645b1a0be34c01ac87c734fcd85ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:24.568268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-6c1c6454-687a-478e-91cf-390fdd709f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:24.599147 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4cd43-5681-4110-a1b0-777bddef5a41 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:24.616258 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df260c-0485-4ede-a5f8-5df148de71cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:24.617224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df260c-0485-4ede-a5f8-5df148de71cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:24.617511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df260c-0485-4ede-a5f8-5df148de71cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:24.617794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df260c-0485-4ede-a5f8-5df148de71cb admin admin] PATH_INFO: `/v3/projects/8aca7a3f3d154c87ad611ce8105aa50f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:24.621274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9df260c-0485-4ede-a5f8-5df148de71cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8aca7a3f3d154c87ad611ce8105aa50f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:24.623329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9df260c-0485-4ede-a5f8-5df148de71cb admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:24.655678 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:24.719061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 922/1832] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jun 16 16:12:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:24.761998 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f9df260c-0485-4ede-a5f8-5df148de71cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8aca7a3f3d154c87ad611ce8105aa50f', 'request_id': 'req-f9df260c-0485-4ede-a5f8-5df148de71cb'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:24.786854 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f9df260c-0485-4ede-a5f8-5df148de71cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8aca7a3f3d154c87ad611ce8105aa50f', 'request_id': 'req-f9df260c-0485-4ede-a5f8-5df148de71cb'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:24.819981 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f9df260c-0485-4ede-a5f8-5df148de71cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8aca7a3f3d154c87ad611ce8105aa50f', 'request_id': 'req-f9df260c-0485-4ede-a5f8-5df148de71cb'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:24.845259 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f9df260c-0485-4ede-a5f8-5df148de71cb admin admin] The token cache is being invalidate because project 8aca7a3f3d154c87ad611ce8105aa50f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:24.845388 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f9df260c-0485-4ede-a5f8-5df148de71cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:24.846525 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f9df260c-0485-4ede-a5f8-5df148de71cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:24.850938 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 911/1833] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:24 2022] DELETE /identity/v3/projects/8aca7a3f3d154c87ad611ce8105aa50f => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:24.857021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156952d-c13f-4567-a13a-a5607a4aed91 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:24.857330 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156952d-c13f-4567-a13a-a5607a4aed91 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:24.857902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156952d-c13f-4567-a13a-a5607a4aed91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:24.886452 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-c156952d-c13f-4567-a13a-a5607a4aed91 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:24.888355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-c156952d-c13f-4567-a13a-a5607a4aed91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:25.026380 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df260c-0485-4ede-a5f8-5df148de71cb admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:25.028520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f9df260c-0485-4ede-a5f8-5df148de71cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:25.042493 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 923/1834] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:25.050604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a28ee-d274-4dcb-81b8-d67cdc4de2fd None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:25.051626 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a28ee-d274-4dcb-81b8-d67cdc4de2fd None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:25.052707 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a28ee-d274-4dcb-81b8-d67cdc4de2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:25.079340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-3e1a28ee-d274-4dcb-81b8-d67cdc4de2fd None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:25.081125 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-3e1a28ee-d274-4dcb-81b8-d67cdc4de2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:25.096233 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 924/1835] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:25.097712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a28ee-d274-4dcb-81b8-d67cdc4de2fd None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:25.111152 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a28ee-d274-4dcb-81b8-d67cdc4de2fd None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:25.117848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e07217-8d52-45db-bd8f-6d8fc8801d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:25.118998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e07217-8d52-45db-bd8f-6d8fc8801d40 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:25.119674 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e07217-8d52-45db-bd8f-6d8fc8801d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:25.121094 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e07217-8d52-45db-bd8f-6d8fc8801d40 admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:25.123342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e07217-8d52-45db-bd8f-6d8fc8801d40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:12:25.124028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e07217-8d52-45db-bd8f-6d8fc8801d40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:25.126450 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e07217-8d52-45db-bd8f-6d8fc8801d40 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:25.143007 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 925/1836] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:12:25.147101 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e07217-8d52-45db-bd8f-6d8fc8801d40 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:25.154915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df260c-0485-4ede-a5f8-5df148de71cb admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:25.165450 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3c65e-3b6e-47d6-8395-838da9ab9829 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:25.167004 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3c65e-3b6e-47d6-8395-838da9ab9829 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:25.167297 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3c65e-3b6e-47d6-8395-838da9ab9829 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:25.167956 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3c65e-3b6e-47d6-8395-838da9ab9829 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:25.168289 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e07217-8d52-45db-bd8f-6d8fc8801d40 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:25.171965 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b3e3c65e-3b6e-47d6-8395-838da9ab9829 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:25.178241 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032e68b-fb96-4dc4-bf65-db17986bc186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:25.179381 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032e68b-fb96-4dc4-bf65-db17986bc186 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:25.180246 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032e68b-fb96-4dc4-bf65-db17986bc186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:25.180843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032e68b-fb96-4dc4-bf65-db17986bc186 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:25.181864 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3032e68b-fb96-4dc4-bf65-db17986bc186 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:25.184351 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3032e68b-fb96-4dc4-bf65-db17986bc186 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:25.189545 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3c65e-3b6e-47d6-8395-838da9ab9829 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:25.192527 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3c65e-3b6e-47d6-8395-838da9ab9829 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:25.216994 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 912/1837] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:12:25.237984 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 926/1838] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:25 2022] POST /identity/v3/projects => generated 383 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:12:25.242015 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3c65e-3b6e-47d6-8395-838da9ab9829 service nova] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:25.274974 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3c65e-3b6e-47d6-8395-838da9ab9829 service nova] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:25.405146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d3c87-3cfc-4c6a-87aa-6775cc838f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:25.405986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d3c87-3cfc-4c6a-87aa-6775cc838f0c admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:25.406351 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d3c87-3cfc-4c6a-87aa-6775cc838f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:25.406880 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d3c87-3cfc-4c6a-87aa-6775cc838f0c admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:25.407904 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d3c87-3cfc-4c6a-87aa-6775cc838f0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:25.410473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d3c87-3cfc-4c6a-87aa-6775cc838f0c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:25.444686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 913/1839] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:25 2022] POST /identity/v3/users => generated 292 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:25.448981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032e68b-fb96-4dc4-bf65-db17986bc186 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:25.477332 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032e68b-fb96-4dc4-bf65-db17986bc186 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:25.495443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa6c57-9f89-4089-9750-c86ef3498645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:25.496534 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa6c57-9f89-4089-9750-c86ef3498645 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:25.497552 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa6c57-9f89-4089-9750-c86ef3498645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:25.498061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa6c57-9f89-4089-9750-c86ef3498645 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:25.499252 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa6c57-9f89-4089-9750-c86ef3498645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:25.501441 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa6c57-9f89-4089-9750-c86ef3498645 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:25.504996 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:25.532103 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 927/1840] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:25.543890 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d3c87-3cfc-4c6a-87aa-6775cc838f0c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:25.560687 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d3c87-3cfc-4c6a-87aa-6775cc838f0c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:25.566157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82e387-1439-4b20-981f-3a4808d25dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:25.567042 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82e387-1439-4b20-981f-3a4808d25dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:25.567424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82e387-1439-4b20-981f-3a4808d25dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:25.567770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82e387-1439-4b20-981f-3a4808d25dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:25.568754 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82e387-1439-4b20-981f-3a4808d25dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:25.574253 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82e387-1439-4b20-981f-3a4808d25dc1 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:25.596592 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 914/1841] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:25.597435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa6c57-9f89-4089-9750-c86ef3498645 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:25.615022 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa6c57-9f89-4089-9750-c86ef3498645 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:25.621017 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24d7c4-df95-43c6-854f-bfff86c7696d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:25.622458 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24d7c4-df95-43c6-854f-bfff86c7696d admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:25.622898 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24d7c4-df95-43c6-854f-bfff86c7696d admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:25.622987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24d7c4-df95-43c6-854f-bfff86c7696d admin admin] PATH_INFO: `/v3/projects/3ccdb67ec1de46ce91b788aefa3f2182/users/27e845541c424176babffa617fa09a6a/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:25.659953 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24d7c4-df95-43c6-854f-bfff86c7696d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ccdb67ec1de46ce91b788aefa3f2182, user_id=27e845541c424176babffa617fa09a6a, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:25.663934 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24d7c4-df95-43c6-854f-bfff86c7696d admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:25.680321 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 928/1842] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:25 2022] PUT /identity/v3/projects/3ccdb67ec1de46ce91b788aefa3f2182/users/27e845541c424176babffa617fa09a6a/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:25.692705 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276c149-71a3-41b1-940f-b51ca69ee17e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:25.692987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276c149-71a3-41b1-940f-b51ca69ee17e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:25.693266 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276c149-71a3-41b1-940f-b51ca69ee17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:25.701427 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:25.728667 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-6276c149-71a3-41b1-940f-b51ca69ee17e None None] MFA Rules not processed for user `27e845541c424176babffa617fa09a6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:25.730294 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6276c149-71a3-41b1-940f-b51ca69ee17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:25.814675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 915/1843] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:25.819507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24d7c4-df95-43c6-854f-bfff86c7696d admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:26.019940 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24d7c4-df95-43c6-854f-bfff86c7696d admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:26.025981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b761a9ac-93f2-4b80-844c-72e571282c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:26.026696 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761a9ac-93f2-4b80-844c-72e571282c63 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:26.026974 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761a9ac-93f2-4b80-844c-72e571282c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:26.027244 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761a9ac-93f2-4b80-844c-72e571282c63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:26.028242 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b761a9ac-93f2-4b80-844c-72e571282c63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:26.030342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b761a9ac-93f2-4b80-844c-72e571282c63 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:26.071560 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 929/1844] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:25 2022] POST /identity/v3/projects => generated 383 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:26.077310 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276c149-71a3-41b1-940f-b51ca69ee17e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:26.093407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276c149-71a3-41b1-940f-b51ca69ee17e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:26.202850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b36d2-c64f-401e-9c64-4681e9f52665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:26.203981 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b36d2-c64f-401e-9c64-4681e9f52665 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:26.204397 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b36d2-c64f-401e-9c64-4681e9f52665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:26.204729 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b36d2-c64f-401e-9c64-4681e9f52665 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:26.206957 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b36d2-c64f-401e-9c64-4681e9f52665 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:26.212006 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b36d2-c64f-401e-9c64-4681e9f52665 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:26.265315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 916/1845] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:26 2022] POST /identity/v3/users => generated 298 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:26.270461 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b761a9ac-93f2-4b80-844c-72e571282c63 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:26.292566 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b761a9ac-93f2-4b80-844c-72e571282c63 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:26.308113 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2492364-fe05-4e0a-b55b-71058162bff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:26.309112 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2492364-fe05-4e0a-b55b-71058162bff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:26.310057 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2492364-fe05-4e0a-b55b-71058162bff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:26.310471 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2492364-fe05-4e0a-b55b-71058162bff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:26.311535 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2492364-fe05-4e0a-b55b-71058162bff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:26.313395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2492364-fe05-4e0a-b55b-71058162bff8 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:26.329886 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 930/1846] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:26.335628 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b36d2-c64f-401e-9c64-4681e9f52665 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:26.351840 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b36d2-c64f-401e-9c64-4681e9f52665 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:26.360231 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e8188-f257-4a16-b66d-50e79b66602f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:26.362238 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e8188-f257-4a16-b66d-50e79b66602f admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:26.362909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e8188-f257-4a16-b66d-50e79b66602f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:26.363315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e8188-f257-4a16-b66d-50e79b66602f admin admin] PATH_INFO: `/v3/projects/5abb815771094d5c9b2f3833298c5b9a/users/bec56597817f4caa9f8a81d5cf844eac/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:26.388914 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e8188-f257-4a16-b66d-50e79b66602f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5abb815771094d5c9b2f3833298c5b9a, user_id=bec56597817f4caa9f8a81d5cf844eac, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:26.392075 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e8188-f257-4a16-b66d-50e79b66602f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:26.409757 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 917/1847] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:26 2022] PUT /identity/v3/projects/5abb815771094d5c9b2f3833298c5b9a/users/bec56597817f4caa9f8a81d5cf844eac/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:26.414122 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2492364-fe05-4e0a-b55b-71058162bff8 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:26.428271 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2492364-fe05-4e0a-b55b-71058162bff8 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:26.616703 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f14838-06e3-4f66-8aa8-65376ac85fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:26.617566 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f14838-06e3-4f66-8aa8-65376ac85fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:26.617848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f14838-06e3-4f66-8aa8-65376ac85fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:26.618320 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f14838-06e3-4f66-8aa8-65376ac85fc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:26.619155 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f14838-06e3-4f66-8aa8-65376ac85fc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:26.621430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f14838-06e3-4f66-8aa8-65376ac85fc5 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:26.637111 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 931/1848] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:26.638877 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e8188-f257-4a16-b66d-50e79b66602f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:26.663100 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e8188-f257-4a16-b66d-50e79b66602f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:26.668576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbd988-2cb8-441d-96ec-a260c037f574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:26.670680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbd988-2cb8-441d-96ec-a260c037f574 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:26.670680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbd988-2cb8-441d-96ec-a260c037f574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:26.670811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbd988-2cb8-441d-96ec-a260c037f574 admin admin] PATH_INFO: `/v3/projects/5abb815771094d5c9b2f3833298c5b9a/users/bec56597817f4caa9f8a81d5cf844eac/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:26.672968 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dbd988-2cb8-441d-96ec-a260c037f574 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5abb815771094d5c9b2f3833298c5b9a, user_id=bec56597817f4caa9f8a81d5cf844eac, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:26.675828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dbd988-2cb8-441d-96ec-a260c037f574 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:26.692950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 918/1849] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:26 2022] PUT /identity/v3/projects/5abb815771094d5c9b2f3833298c5b9a/users/bec56597817f4caa9f8a81d5cf844eac/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:26.714417 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534df6e-ca6d-48a5-ad4a-7406b0afcc2a None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:26.714713 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534df6e-ca6d-48a5-ad4a-7406b0afcc2a None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:26.715011 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534df6e-ca6d-48a5-ad4a-7406b0afcc2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:26.744790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-f534df6e-ca6d-48a5-ad4a-7406b0afcc2a None None] MFA Rules not processed for user `bec56597817f4caa9f8a81d5cf844eac`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:26.746040 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-f534df6e-ca6d-48a5-ad4a-7406b0afcc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:26.788737 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:26.837383 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 932/1850] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:26.848755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbd988-2cb8-441d-96ec-a260c037f574 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:26.850819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-51dbd988-2cb8-441d-96ec-a260c037f574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:26.900226 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:26.912659 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbd988-2cb8-441d-96ec-a260c037f574 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:26.917427 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c65bf-11a7-44ca-b605-a8ec3971c629 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:26.918141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c65bf-11a7-44ca-b605-a8ec3971c629 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:26.918421 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c65bf-11a7-44ca-b605-a8ec3971c629 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:26.918797 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c65bf-11a7-44ca-b605-a8ec3971c629 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:26.977182 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c65bf-11a7-44ca-b605-a8ec3971c629 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:26.978649 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c65bf-11a7-44ca-b605-a8ec3971c629 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:26.988998 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 919/1851] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:27.161121 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534df6e-ca6d-48a5-ad4a-7406b0afcc2a None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:27.172296 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534df6e-ca6d-48a5-ad4a-7406b0afcc2a None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:27.176004 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea6d10a-32ab-4755-9f8e-2907b005a23e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:27.176653 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6d10a-32ab-4755-9f8e-2907b005a23e service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:27.176932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6d10a-32ab-4755-9f8e-2907b005a23e service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:27.177206 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6d10a-32ab-4755-9f8e-2907b005a23e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:27.179078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-6ea6d10a-32ab-4755-9f8e-2907b005a23e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:27.265342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea6d10a-32ab-4755-9f8e-2907b005a23e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:27.267152 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea6d10a-32ab-4755-9f8e-2907b005a23e service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:27.280020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 933/1852] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:27 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:32.432131 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c65bf-11a7-44ca-b605-a8ec3971c629 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:32.435803 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ab3c65bf-11a7-44ca-b605-a8ec3971c629 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:32.663052 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c65bf-11a7-44ca-b605-a8ec3971c629 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:32.689654 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa4789-7f45-4161-adea-2fa33eb2d87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:32.690796 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa4789-7f45-4161-adea-2fa33eb2d87c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:32.691742 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa4789-7f45-4161-adea-2fa33eb2d87c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:32.692995 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa4789-7f45-4161-adea-2fa33eb2d87c admin admin] PATH_INFO: `/v3/users/d795c97cb7fc41ebab6d0cf575ff7e9a` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:32.702940 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa4789-7f45-4161-adea-2fa33eb2d87c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d795c97cb7fc41ebab6d0cf575ff7e9a)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:32.707002 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa4789-7f45-4161-adea-2fa33eb2d87c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:32.876066 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4aaa4789-7f45-4161-adea-2fa33eb2d87c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd795c97cb7fc41ebab6d0cf575ff7e9a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:32.911166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504bb899-6439-4026-91ac-afa61befd45f None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:32.911748 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504bb899-6439-4026-91ac-afa61befd45f None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:32.911748 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504bb899-6439-4026-91ac-afa61befd45f None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:32.913323 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 934/1853] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jun 16 16:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:12:32.920627 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4aaa4789-7f45-4161-adea-2fa33eb2d87c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd795c97cb7fc41ebab6d0cf575ff7e9a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:32.929281 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d83cf-f462-4445-ba45-e40415d34953 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:32.929837 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d83cf-f462-4445-ba45-e40415d34953 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:32.930372 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d83cf-f462-4445-ba45-e40415d34953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:32.933620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4aaa4789-7f45-4161-adea-2fa33eb2d87c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd795c97cb7fc41ebab6d0cf575ff7e9a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:32.958433 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-320d83cf-f462-4445-ba45-e40415d34953 None None] MFA Rules not processed for user `c3645b1a0be34c01ac87c734fcd85ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:32.959687 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-320d83cf-f462-4445-ba45-e40415d34953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:32.971055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4aaa4789-7f45-4161-adea-2fa33eb2d87c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd795c97cb7fc41ebab6d0cf575ff7e9a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:32.973011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4aaa4789-7f45-4161-adea-2fa33eb2d87c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd795c97cb7fc41ebab6d0cf575ff7e9a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:33.009960 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 920/1854] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:32 2022] DELETE /identity/v3/users/d795c97cb7fc41ebab6d0cf575ff7e9a => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:33.021130 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75df1d0-d21f-4b2b-9a3a-9a4ea68190fd None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:33.021590 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75df1d0-d21f-4b2b-9a3a-9a4ea68190fd None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:33.021995 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75df1d0-d21f-4b2b-9a3a-9a4ea68190fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:33.069452 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b75df1d0-d21f-4b2b-9a3a-9a4ea68190fd None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:33.071926 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b75df1d0-d21f-4b2b-9a3a-9a4ea68190fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:33.095945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:33.212148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 935/1855] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jun 16 16:12:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:12:33.230855 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320d83cf-f462-4445-ba45-e40415d34953 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:33.233783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-320d83cf-f462-4445-ba45-e40415d34953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:33.246894 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 921/1856] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:33.261193 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320d83cf-f462-4445-ba45-e40415d34953 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:33.304658 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75df1d0-d21f-4b2b-9a3a-9a4ea68190fd None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:33.306601 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b75df1d0-d21f-4b2b-9a3a-9a4ea68190fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:33.401056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:33.432912 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20450cb1-2596-4439-91e3-6550216b7f48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:33.434217 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20450cb1-2596-4439-91e3-6550216b7f48 service nova] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:33.434721 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20450cb1-2596-4439-91e3-6550216b7f48 service nova] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:33.435112 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20450cb1-2596-4439-91e3-6550216b7f48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:33.455317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20450cb1-2596-4439-91e3-6550216b7f48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:33.459004 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20450cb1-2596-4439-91e3-6550216b7f48 service nova] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:33.480578 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75df1d0-d21f-4b2b-9a3a-9a4ea68190fd None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:33.486437 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 936/1857] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:33.499204 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfeeb8-3e09-4bf3-bc9f-e33a4b1db72e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:33.500590 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfeeb8-3e09-4bf3-bc9f-e33a4b1db72e service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:33.501235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfeeb8-3e09-4bf3-bc9f-e33a4b1db72e service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:33.501960 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfeeb8-3e09-4bf3-bc9f-e33a4b1db72e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:33.537656 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfeeb8-3e09-4bf3-bc9f-e33a4b1db72e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:33.545201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfeeb8-3e09-4bf3-bc9f-e33a4b1db72e service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:33.574360 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 922/1858] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:34.011068 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20450cb1-2596-4439-91e3-6550216b7f48 service nova] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:34.012547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-20450cb1-2596-4439-91e3-6550216b7f48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:34.155873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20450cb1-2596-4439-91e3-6550216b7f48 service nova] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:34.168215 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e41f43-6b47-42fb-8a5c-f8a33fe8f915 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:34.169703 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e41f43-6b47-42fb-8a5c-f8a33fe8f915 service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:34.171847 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e41f43-6b47-42fb-8a5c-f8a33fe8f915 service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:34.172507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e41f43-6b47-42fb-8a5c-f8a33fe8f915 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:34.179531 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-10e41f43-6b47-42fb-8a5c-f8a33fe8f915 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:34.351447 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e41f43-6b47-42fb-8a5c-f8a33fe8f915 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:34.352973 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e41f43-6b47-42fb-8a5c-f8a33fe8f915 service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:34.390501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 937/1859] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:34 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:35.425128 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfeeb8-3e09-4bf3-bc9f-e33a4b1db72e service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:35.426701 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-33dfeeb8-3e09-4bf3-bc9f-e33a4b1db72e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:35.442035 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfeeb8-3e09-4bf3-bc9f-e33a4b1db72e service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:35.452514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b836e0-25c7-4452-9196-2660c83d1cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:35.453489 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b836e0-25c7-4452-9196-2660c83d1cd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:35.453772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b836e0-25c7-4452-9196-2660c83d1cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:35.454220 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b836e0-25c7-4452-9196-2660c83d1cd9 admin admin] PATH_INFO: `/v3/projects/1b6cfdd94e2d43cab5b14d1027d243ed` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:35.455455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b836e0-25c7-4452-9196-2660c83d1cd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b6cfdd94e2d43cab5b14d1027d243ed)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:35.457447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b836e0-25c7-4452-9196-2660c83d1cd9 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:35.515942 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c2b836e0-25c7-4452-9196-2660c83d1cd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b6cfdd94e2d43cab5b14d1027d243ed', 'request_id': 'req-c2b836e0-25c7-4452-9196-2660c83d1cd9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:35.522108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c2b836e0-25c7-4452-9196-2660c83d1cd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b6cfdd94e2d43cab5b14d1027d243ed', 'request_id': 'req-c2b836e0-25c7-4452-9196-2660c83d1cd9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:35.531973 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c2b836e0-25c7-4452-9196-2660c83d1cd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b6cfdd94e2d43cab5b14d1027d243ed', 'request_id': 'req-c2b836e0-25c7-4452-9196-2660c83d1cd9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:35.547046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c2b836e0-25c7-4452-9196-2660c83d1cd9 admin admin] The token cache is being invalidate because project 1b6cfdd94e2d43cab5b14d1027d243ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:35.547294 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c2b836e0-25c7-4452-9196-2660c83d1cd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:35.548018 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c2b836e0-25c7-4452-9196-2660c83d1cd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:35.549755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 923/1860] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:35 2022] DELETE /identity/v3/projects/1b6cfdd94e2d43cab5b14d1027d243ed => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:35.553983 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e41f43-6b47-42fb-8a5c-f8a33fe8f915 service cinder] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:35.566067 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e41f43-6b47-42fb-8a5c-f8a33fe8f915 service cinder] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:35.635338 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aca716-24fa-4993-8bcd-26da991d2be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:35.636574 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aca716-24fa-4993-8bcd-26da991d2be6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:35.636885 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aca716-24fa-4993-8bcd-26da991d2be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:35.637366 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aca716-24fa-4993-8bcd-26da991d2be6 admin admin] PATH_INFO: `/v3/users/db0cd76592b44dc18c53ed77b57be642` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:35.639100 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aca716-24fa-4993-8bcd-26da991d2be6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db0cd76592b44dc18c53ed77b57be642)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:35.641397 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aca716-24fa-4993-8bcd-26da991d2be6 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:35.685572 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:35.705598 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-95aca716-24fa-4993-8bcd-26da991d2be6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db0cd76592b44dc18c53ed77b57be642'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:35.713354 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-95aca716-24fa-4993-8bcd-26da991d2be6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db0cd76592b44dc18c53ed77b57be642'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:35.727684 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-95aca716-24fa-4993-8bcd-26da991d2be6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db0cd76592b44dc18c53ed77b57be642'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:35.728454 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-95aca716-24fa-4993-8bcd-26da991d2be6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db0cd76592b44dc18c53ed77b57be642'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:35.738357 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-95aca716-24fa-4993-8bcd-26da991d2be6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db0cd76592b44dc18c53ed77b57be642'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:35.745535 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 938/1861] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:35 2022] DELETE /identity/v3/users/db0cd76592b44dc18c53ed77b57be642 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:36.142777 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b836e0-25c7-4452-9196-2660c83d1cd9 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:36.146085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c2b836e0-25c7-4452-9196-2660c83d1cd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:36.172531 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b836e0-25c7-4452-9196-2660c83d1cd9 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:36.261242 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:36.287471 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7ecf8-5f9e-44dd-8c82-75a9ebc99718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:36.288382 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7ecf8-5f9e-44dd-8c82-75a9ebc99718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:36.288668 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7ecf8-5f9e-44dd-8c82-75a9ebc99718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:36.289024 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7ecf8-5f9e-44dd-8c82-75a9ebc99718 admin admin] PATH_INFO: `/v3/projects/c16ff4583f384dd48b6544041cff397d` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:36.290576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e7ecf8-5f9e-44dd-8c82-75a9ebc99718 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c16ff4583f384dd48b6544041cff397d)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:36.292557 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e7ecf8-5f9e-44dd-8c82-75a9ebc99718 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:36.376058 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-24e7ecf8-5f9e-44dd-8c82-75a9ebc99718 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c16ff4583f384dd48b6544041cff397d', 'request_id': 'req-24e7ecf8-5f9e-44dd-8c82-75a9ebc99718'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:36.383606 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-24e7ecf8-5f9e-44dd-8c82-75a9ebc99718 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c16ff4583f384dd48b6544041cff397d', 'request_id': 'req-24e7ecf8-5f9e-44dd-8c82-75a9ebc99718'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:36.392935 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-24e7ecf8-5f9e-44dd-8c82-75a9ebc99718 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c16ff4583f384dd48b6544041cff397d', 'request_id': 'req-24e7ecf8-5f9e-44dd-8c82-75a9ebc99718'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:36.409858 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-24e7ecf8-5f9e-44dd-8c82-75a9ebc99718 admin admin] The token cache is being invalidate because project c16ff4583f384dd48b6544041cff397d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:36.410251 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-24e7ecf8-5f9e-44dd-8c82-75a9ebc99718 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:36.410867 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-24e7ecf8-5f9e-44dd-8c82-75a9ebc99718 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:36.412517 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 924/1862] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:36 2022] DELETE /identity/v3/projects/c16ff4583f384dd48b6544041cff397d => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:36.418214 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ef1e7-f2fd-4e4c-85b1-be2f0897199c None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:36.420061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ef1e7-f2fd-4e4c-85b1-be2f0897199c None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:36.420360 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ef1e7-f2fd-4e4c-85b1-be2f0897199c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:36.445642 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-e35ef1e7-f2fd-4e4c-85b1-be2f0897199c None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:36.446979 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-e35ef1e7-f2fd-4e4c-85b1-be2f0897199c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:36.531078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 939/1863] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:36.538802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e11fdc-2086-4e7b-a061-b475ad696153 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:36.539506 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e11fdc-2086-4e7b-a061-b475ad696153 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:36.539506 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e11fdc-2086-4e7b-a061-b475ad696153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:36.563071 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-74e11fdc-2086-4e7b-a061-b475ad696153 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:36.564803 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-74e11fdc-2086-4e7b-a061-b475ad696153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:36.574282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 925/1864] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:36.579396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ef1e7-f2fd-4e4c-85b1-be2f0897199c None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:36.590716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ef1e7-f2fd-4e4c-85b1-be2f0897199c None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:36.596025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fdc6d3-6e8d-4978-8e1a-b4d78f3d2573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:36.596609 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdc6d3-6e8d-4978-8e1a-b4d78f3d2573 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:36.596948 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdc6d3-6e8d-4978-8e1a-b4d78f3d2573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:36.597256 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdc6d3-6e8d-4978-8e1a-b4d78f3d2573 admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:36.598060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fdc6d3-6e8d-4978-8e1a-b4d78f3d2573 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:12:36.598374 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fdc6d3-6e8d-4978-8e1a-b4d78f3d2573 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:36.600058 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fdc6d3-6e8d-4978-8e1a-b4d78f3d2573 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:36.620527 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 940/1865] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:12:36.620969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e11fdc-2086-4e7b-a061-b475ad696153 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:36.632796 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e11fdc-2086-4e7b-a061-b475ad696153 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:36.638596 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb163194-a677-4477-9571-e8ef2b6a5be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:36.639303 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb163194-a677-4477-9571-e8ef2b6a5be0 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:36.639589 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb163194-a677-4477-9571-e8ef2b6a5be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:36.639867 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb163194-a677-4477-9571-e8ef2b6a5be0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:36.640597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb163194-a677-4477-9571-e8ef2b6a5be0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:36.642663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb163194-a677-4477-9571-e8ef2b6a5be0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:36.677463 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 926/1866] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:36 2022] POST /identity/v3/projects => generated 407 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:12:36.681665 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fdc6d3-6e8d-4978-8e1a-b4d78f3d2573 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:36.692782 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fdc6d3-6e8d-4978-8e1a-b4d78f3d2573 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:36.795167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd01b14-8dea-4267-b4b1-442d5b37765e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:36.796587 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd01b14-8dea-4267-b4b1-442d5b37765e admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:36.796587 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd01b14-8dea-4267-b4b1-442d5b37765e admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:36.796587 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd01b14-8dea-4267-b4b1-442d5b37765e admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:36.797323 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd01b14-8dea-4267-b4b1-442d5b37765e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:36.799361 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd01b14-8dea-4267-b4b1-442d5b37765e admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:36.823149 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 941/1867] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:36 2022] POST /identity/v3/users => generated 304 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:12:36.828144 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb163194-a677-4477-9571-e8ef2b6a5be0 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:36.845446 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb163194-a677-4477-9571-e8ef2b6a5be0 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:36.850263 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea878370-9c94-4e80-90ad-836d04a5188a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:36.850945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea878370-9c94-4e80-90ad-836d04a5188a admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:36.851225 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea878370-9c94-4e80-90ad-836d04a5188a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:36.851498 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea878370-9c94-4e80-90ad-836d04a5188a admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:36.852234 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea878370-9c94-4e80-90ad-836d04a5188a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:36.853859 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea878370-9c94-4e80-90ad-836d04a5188a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:36.866103 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 927/1868] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:36.870436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd01b14-8dea-4267-b4b1-442d5b37765e admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:36.881655 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd01b14-8dea-4267-b4b1-442d5b37765e admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:36.886823 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf38aa6-2745-4ca6-8c87-c56904b0c1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:36.887554 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf38aa6-2745-4ca6-8c87-c56904b0c1de admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:36.887836 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf38aa6-2745-4ca6-8c87-c56904b0c1de admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:36.888123 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf38aa6-2745-4ca6-8c87-c56904b0c1de admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:36.888863 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf38aa6-2745-4ca6-8c87-c56904b0c1de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:36.890543 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf38aa6-2745-4ca6-8c87-c56904b0c1de admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:36.901746 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 942/1869] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:36.906353 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea878370-9c94-4e80-90ad-836d04a5188a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:36.916805 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea878370-9c94-4e80-90ad-836d04a5188a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:36.922702 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d76c2-56d4-4307-8b16-70f7e6169be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:36.923777 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d76c2-56d4-4307-8b16-70f7e6169be5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:36.924072 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d76c2-56d4-4307-8b16-70f7e6169be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:36.924353 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d76c2-56d4-4307-8b16-70f7e6169be5 admin admin] PATH_INFO: `/v3/projects/85ff1f13e634456d92e39796a86e9419/users/c265e168184342138a0909202440c560/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:36.942007 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d76c2-56d4-4307-8b16-70f7e6169be5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85ff1f13e634456d92e39796a86e9419, user_id=c265e168184342138a0909202440c560, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:36.944725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d76c2-56d4-4307-8b16-70f7e6169be5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:36.960102 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 928/1870] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:36 2022] PUT /identity/v3/projects/85ff1f13e634456d92e39796a86e9419/users/c265e168184342138a0909202440c560/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:36.972079 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6930834-6bff-4967-acfa-6fd13a338834 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:36.972281 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6930834-6bff-4967-acfa-6fd13a338834 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:36.972563 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6930834-6bff-4967-acfa-6fd13a338834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:36.999623 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:37.003749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-a6930834-6bff-4967-acfa-6fd13a338834 None None] MFA Rules not processed for user `c265e168184342138a0909202440c560`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:37.005693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-a6930834-6bff-4967-acfa-6fd13a338834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:37.078798 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 943/1871] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:36 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:37.083254 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d76c2-56d4-4307-8b16-70f7e6169be5 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:37.155102 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d76c2-56d4-4307-8b16-70f7e6169be5 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:37.160138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1bd00-0b6b-4de0-bbc1-05a0fcd81f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:37.160827 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1bd00-0b6b-4de0-bbc1-05a0fcd81f60 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:37.161109 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1bd00-0b6b-4de0-bbc1-05a0fcd81f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:37.161382 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1bd00-0b6b-4de0-bbc1-05a0fcd81f60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:37.162142 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d1bd00-0b6b-4de0-bbc1-05a0fcd81f60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:37.164107 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d1bd00-0b6b-4de0-bbc1-05a0fcd81f60 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:37.182939 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:37.201129 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 929/1872] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:37 2022] POST /identity/v3/projects => generated 407 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:37.205842 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6930834-6bff-4967-acfa-6fd13a338834 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:37.219353 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6930834-6bff-4967-acfa-6fd13a338834 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:37.304067 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e96532-8150-4544-94f8-4bf80326b015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:37.304901 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e96532-8150-4544-94f8-4bf80326b015 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:37.305276 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e96532-8150-4544-94f8-4bf80326b015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:37.305620 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e96532-8150-4544-94f8-4bf80326b015 admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:37.306548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e96532-8150-4544-94f8-4bf80326b015 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:37.308761 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e96532-8150-4544-94f8-4bf80326b015 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:37.333512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 944/1873] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:37 2022] POST /identity/v3/users => generated 310 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:37.337965 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1bd00-0b6b-4de0-bbc1-05a0fcd81f60 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:37.350437 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1bd00-0b6b-4de0-bbc1-05a0fcd81f60 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:37.357824 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea472775-5705-4b94-a379-59884513e792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:37.358592 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea472775-5705-4b94-a379-59884513e792 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:37.358874 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea472775-5705-4b94-a379-59884513e792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:37.359157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea472775-5705-4b94-a379-59884513e792 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:37.359990 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea472775-5705-4b94-a379-59884513e792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:37.361867 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea472775-5705-4b94-a379-59884513e792 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:37.379787 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 930/1874] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:37.379997 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e96532-8150-4544-94f8-4bf80326b015 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:37.391923 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e96532-8150-4544-94f8-4bf80326b015 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:37.404270 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea06d89-3ae2-4968-8fe5-4fa7c7fb2a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:37.405730 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea06d89-3ae2-4968-8fe5-4fa7c7fb2a04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:37.405730 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea06d89-3ae2-4968-8fe5-4fa7c7fb2a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:37.405730 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea06d89-3ae2-4968-8fe5-4fa7c7fb2a04 admin admin] PATH_INFO: `/v3/projects/3dc7ab8276a944fab6ab89374628d2b3/users/e68bec05f04049a8976c63d17a711900/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:37.426830 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea06d89-3ae2-4968-8fe5-4fa7c7fb2a04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dc7ab8276a944fab6ab89374628d2b3, user_id=e68bec05f04049a8976c63d17a711900, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:37.429625 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea06d89-3ae2-4968-8fe5-4fa7c7fb2a04 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:37.445501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 945/1875] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:37 2022] PUT /identity/v3/projects/3dc7ab8276a944fab6ab89374628d2b3/users/e68bec05f04049a8976c63d17a711900/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:37.449376 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea472775-5705-4b94-a379-59884513e792 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:37.464966 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea472775-5705-4b94-a379-59884513e792 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:37.554969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8959bc4-d4ed-4578-8980-8b507d047192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:37.555834 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8959bc4-d4ed-4578-8980-8b507d047192 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:37.556126 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8959bc4-d4ed-4578-8980-8b507d047192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:37.556430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8959bc4-d4ed-4578-8980-8b507d047192 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:37.557223 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8959bc4-d4ed-4578-8980-8b507d047192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:37.559651 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8959bc4-d4ed-4578-8980-8b507d047192 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:37.575037 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 931/1876] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:12:37.586617 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea06d89-3ae2-4968-8fe5-4fa7c7fb2a04 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:37.604327 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea06d89-3ae2-4968-8fe5-4fa7c7fb2a04 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:37.612456 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6daf984-2ded-4206-b638-45e951adae3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:37.614495 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6daf984-2ded-4206-b638-45e951adae3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:37.614945 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6daf984-2ded-4206-b638-45e951adae3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:37.615457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6daf984-2ded-4206-b638-45e951adae3d admin admin] PATH_INFO: `/v3/projects/3dc7ab8276a944fab6ab89374628d2b3/users/e68bec05f04049a8976c63d17a711900/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:37.619384 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6daf984-2ded-4206-b638-45e951adae3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dc7ab8276a944fab6ab89374628d2b3, user_id=e68bec05f04049a8976c63d17a711900, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:37.624949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6daf984-2ded-4206-b638-45e951adae3d admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:37.646615 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 946/1877] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:37 2022] PUT /identity/v3/projects/3dc7ab8276a944fab6ab89374628d2b3/users/e68bec05f04049a8976c63d17a711900/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:37.661831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481e5b6-c5fb-4a7a-8c18-bf5e8dd8398d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:37.662363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481e5b6-c5fb-4a7a-8c18-bf5e8dd8398d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:37.662870 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481e5b6-c5fb-4a7a-8c18-bf5e8dd8398d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:37.718919 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8481e5b6-c5fb-4a7a-8c18-bf5e8dd8398d None None] MFA Rules not processed for user `e68bec05f04049a8976c63d17a711900`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:37.720384 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8481e5b6-c5fb-4a7a-8c18-bf5e8dd8398d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:37.846353 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 932/1878] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:37 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:12:37.859691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6daf984-2ded-4206-b638-45e951adae3d admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:37.862471 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-d6daf984-2ded-4206-b638-45e951adae3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:37.941787 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6daf984-2ded-4206-b638-45e951adae3d admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:37.946890 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696c90a-38e5-4897-a5a5-f8848e397f9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:37.947940 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696c90a-38e5-4897-a5a5-f8848e397f9d service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:37.948453 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696c90a-38e5-4897-a5a5-f8848e397f9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:37.948935 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696c90a-38e5-4897-a5a5-f8848e397f9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:37.964933 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4696c90a-38e5-4897-a5a5-f8848e397f9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:37.967497 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4696c90a-38e5-4897-a5a5-f8848e397f9d service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:37.972444 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:37.981270 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 947/1879] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:37 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:38.395582 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8481e5b6-c5fb-4a7a-8c18-bf5e8dd8398d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:38.399374 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8481e5b6-c5fb-4a7a-8c18-bf5e8dd8398d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:38.432323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8481e5b6-c5fb-4a7a-8c18-bf5e8dd8398d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:38.442621 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777a77f-373b-4b96-9200-42813688879a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:38.445179 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777a77f-373b-4b96-9200-42813688879a service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:38.445851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777a77f-373b-4b96-9200-42813688879a service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:38.446391 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777a77f-373b-4b96-9200-42813688879a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:38.599873 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a777a77f-373b-4b96-9200-42813688879a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:38.601585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a777a77f-373b-4b96-9200-42813688879a service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:38.619327 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 933/1880] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:38 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:40.349239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696c90a-38e5-4897-a5a5-f8848e397f9d service cinder] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:40.350725 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-4696c90a-38e5-4897-a5a5-f8848e397f9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:40.468217 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696c90a-38e5-4897-a5a5-f8848e397f9d service cinder] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:40.475878 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559592a3-e832-44cc-ba65-1efa0eda6fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:40.477621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559592a3-e832-44cc-ba65-1efa0eda6fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:40.478030 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559592a3-e832-44cc-ba65-1efa0eda6fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:40.478389 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559592a3-e832-44cc-ba65-1efa0eda6fb6 admin admin] PATH_INFO: `/v3/users/8fb1375fb4a9462eb7b2ec1ae0ff7836` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:40.481416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559592a3-e832-44cc-ba65-1efa0eda6fb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fb1375fb4a9462eb7b2ec1ae0ff7836)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:40.483770 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559592a3-e832-44cc-ba65-1efa0eda6fb6 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:40.593010 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-559592a3-e832-44cc-ba65-1efa0eda6fb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fb1375fb4a9462eb7b2ec1ae0ff7836'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:40.605608 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-559592a3-e832-44cc-ba65-1efa0eda6fb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fb1375fb4a9462eb7b2ec1ae0ff7836'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:40.631014 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-559592a3-e832-44cc-ba65-1efa0eda6fb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fb1375fb4a9462eb7b2ec1ae0ff7836'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:40.633125 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-559592a3-e832-44cc-ba65-1efa0eda6fb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fb1375fb4a9462eb7b2ec1ae0ff7836'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:40.655947 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-559592a3-e832-44cc-ba65-1efa0eda6fb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fb1375fb4a9462eb7b2ec1ae0ff7836'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:40.667350 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 948/1881] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:40 2022] DELETE /identity/v3/users/8fb1375fb4a9462eb7b2ec1ae0ff7836 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:40.677466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ceb3fe-0528-4d4d-ab82-53ee60dd6122 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:40.678069 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ceb3fe-0528-4d4d-ab82-53ee60dd6122 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:40.678421 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ceb3fe-0528-4d4d-ab82-53ee60dd6122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:40.690241 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:40.722432 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-32ceb3fe-0528-4d4d-ab82-53ee60dd6122 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:40.724019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-32ceb3fe-0528-4d4d-ab82-53ee60dd6122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:40.896239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 934/1882] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:40.924874 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559592a3-e832-44cc-ba65-1efa0eda6fb6 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:40.927843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-559592a3-e832-44cc-ba65-1efa0eda6fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:41.151071 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559592a3-e832-44cc-ba65-1efa0eda6fb6 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:41.164302 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a202c7-64d7-44c2-aad2-0dd13e49339c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:41.165061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a202c7-64d7-44c2-aad2-0dd13e49339c service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:41.165381 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a202c7-64d7-44c2-aad2-0dd13e49339c service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:41.165653 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a202c7-64d7-44c2-aad2-0dd13e49339c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:41.204688 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a202c7-64d7-44c2-aad2-0dd13e49339c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:41.206306 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a202c7-64d7-44c2-aad2-0dd13e49339c service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:41.267168 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 949/1883] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:42.201855 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ceb3fe-0528-4d4d-ab82-53ee60dd6122 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:42.205972 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-32ceb3fe-0528-4d4d-ab82-53ee60dd6122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:42.251975 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ceb3fe-0528-4d4d-ab82-53ee60dd6122 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:42.258954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b4906-5161-4e5c-8b0a-a291ac2f5877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:42.260289 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b4906-5161-4e5c-8b0a-a291ac2f5877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:42.260778 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b4906-5161-4e5c-8b0a-a291ac2f5877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:42.261557 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b4906-5161-4e5c-8b0a-a291ac2f5877 admin admin] PATH_INFO: `/v3/projects/901c3a6a0a8e4034ad0e23a1b57c7bf5` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:42.264235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b4906-5161-4e5c-8b0a-a291ac2f5877 admin admin] RBAC: Authorizing `identity:delete_project(project_id=901c3a6a0a8e4034ad0e23a1b57c7bf5)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:42.266345 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b4906-5161-4e5c-8b0a-a291ac2f5877 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:42.349038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a43b4906-5161-4e5c-8b0a-a291ac2f5877 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '901c3a6a0a8e4034ad0e23a1b57c7bf5', 'request_id': 'req-a43b4906-5161-4e5c-8b0a-a291ac2f5877'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:42.355260 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a43b4906-5161-4e5c-8b0a-a291ac2f5877 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '901c3a6a0a8e4034ad0e23a1b57c7bf5', 'request_id': 'req-a43b4906-5161-4e5c-8b0a-a291ac2f5877'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:42.364724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a43b4906-5161-4e5c-8b0a-a291ac2f5877 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '901c3a6a0a8e4034ad0e23a1b57c7bf5', 'request_id': 'req-a43b4906-5161-4e5c-8b0a-a291ac2f5877'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:42.380103 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a43b4906-5161-4e5c-8b0a-a291ac2f5877 admin admin] The token cache is being invalidate because project 901c3a6a0a8e4034ad0e23a1b57c7bf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:42.380476 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a43b4906-5161-4e5c-8b0a-a291ac2f5877 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:42.381099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a43b4906-5161-4e5c-8b0a-a291ac2f5877 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:42.383642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 935/1884] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:42 2022] DELETE /identity/v3/projects/901c3a6a0a8e4034ad0e23a1b57c7bf5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:42.388009 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a202c7-64d7-44c2-aad2-0dd13e49339c service neutron] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:42.399303 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a202c7-64d7-44c2-aad2-0dd13e49339c service neutron] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:42.420651 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:42.487320 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf2827-aa5c-4ac6-b2ca-c47b080c4b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:42.488440 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf2827-aa5c-4ac6-b2ca-c47b080c4b42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:42.489034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf2827-aa5c-4ac6-b2ca-c47b080c4b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:42.489738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf2827-aa5c-4ac6-b2ca-c47b080c4b42 admin admin] PATH_INFO: `/v3/users/9fcff3007a1948c8bb9b3641e42f7392` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:42.491528 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf2827-aa5c-4ac6-b2ca-c47b080c4b42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fcff3007a1948c8bb9b3641e42f7392)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:42.494573 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf2827-aa5c-4ac6-b2ca-c47b080c4b42 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:42.571209 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-bfdf2827-aa5c-4ac6-b2ca-c47b080c4b42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fcff3007a1948c8bb9b3641e42f7392'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:42.582317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-bfdf2827-aa5c-4ac6-b2ca-c47b080c4b42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fcff3007a1948c8bb9b3641e42f7392'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:42.602887 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-bfdf2827-aa5c-4ac6-b2ca-c47b080c4b42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fcff3007a1948c8bb9b3641e42f7392'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:42.603842 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-bfdf2827-aa5c-4ac6-b2ca-c47b080c4b42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fcff3007a1948c8bb9b3641e42f7392'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:42.614159 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-bfdf2827-aa5c-4ac6-b2ca-c47b080c4b42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fcff3007a1948c8bb9b3641e42f7392'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:42.623099 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 950/1885] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:42 2022] DELETE /identity/v3/users/9fcff3007a1948c8bb9b3641e42f7392 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:12:43.046264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b4906-5161-4e5c-8b0a-a291ac2f5877 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:43.048351 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a43b4906-5161-4e5c-8b0a-a291ac2f5877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:43.071197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b4906-5161-4e5c-8b0a-a291ac2f5877 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:43.161915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb56cc-6ad4-4c71-95cc-414c41dd77be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:43.162855 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb56cc-6ad4-4c71-95cc-414c41dd77be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:43.163136 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb56cc-6ad4-4c71-95cc-414c41dd77be admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:43.163522 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb56cc-6ad4-4c71-95cc-414c41dd77be admin admin] PATH_INFO: `/v3/projects/827c811c01da43459a02f59586b36d44` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:43.165041 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eb56cc-6ad4-4c71-95cc-414c41dd77be admin admin] RBAC: Authorizing `identity:delete_project(project_id=827c811c01da43459a02f59586b36d44)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:43.167251 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eb56cc-6ad4-4c71-95cc-414c41dd77be admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:43.213979 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:43.234484 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a9eb56cc-6ad4-4c71-95cc-414c41dd77be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '827c811c01da43459a02f59586b36d44', 'request_id': 'req-a9eb56cc-6ad4-4c71-95cc-414c41dd77be'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:43.247843 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a9eb56cc-6ad4-4c71-95cc-414c41dd77be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '827c811c01da43459a02f59586b36d44', 'request_id': 'req-a9eb56cc-6ad4-4c71-95cc-414c41dd77be'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:43.258195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a9eb56cc-6ad4-4c71-95cc-414c41dd77be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '827c811c01da43459a02f59586b36d44', 'request_id': 'req-a9eb56cc-6ad4-4c71-95cc-414c41dd77be'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:43.273768 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a9eb56cc-6ad4-4c71-95cc-414c41dd77be admin admin] The token cache is being invalidate because project 827c811c01da43459a02f59586b36d44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:43.274343 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a9eb56cc-6ad4-4c71-95cc-414c41dd77be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:43.275038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a9eb56cc-6ad4-4c71-95cc-414c41dd77be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:43.276643 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 936/1886] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:43 2022] DELETE /identity/v3/projects/827c811c01da43459a02f59586b36d44 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:12:43.282452 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed607452-ee95-4fce-9b65-b495960c164c None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:43.282774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed607452-ee95-4fce-9b65-b495960c164c None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:43.283106 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed607452-ee95-4fce-9b65-b495960c164c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:43.312391 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-ed607452-ee95-4fce-9b65-b495960c164c None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:43.313865 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-ed607452-ee95-4fce-9b65-b495960c164c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:43.431247 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 951/1887] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:12:43.441427 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41b566-c709-43d4-99d9-ec84ef81c222 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:43.441826 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41b566-c709-43d4-99d9-ec84ef81c222 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:43.442310 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41b566-c709-43d4-99d9-ec84ef81c222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:43.473125 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-3c41b566-c709-43d4-99d9-ec84ef81c222 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:43.475150 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3c41b566-c709-43d4-99d9-ec84ef81c222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:43.491008 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 937/1888] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:43.495629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed607452-ee95-4fce-9b65-b495960c164c None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:43.511247 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed607452-ee95-4fce-9b65-b495960c164c None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:43.517040 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42917781-d91b-4215-b0d0-5e6a14704397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:43.517774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42917781-d91b-4215-b0d0-5e6a14704397 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:43.518095 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42917781-d91b-4215-b0d0-5e6a14704397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:43.518377 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42917781-d91b-4215-b0d0-5e6a14704397 admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:43.519207 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42917781-d91b-4215-b0d0-5e6a14704397 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:12:43.519733 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42917781-d91b-4215-b0d0-5e6a14704397 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:43.522094 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42917781-d91b-4215-b0d0-5e6a14704397 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:43.546356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 952/1889] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:12:43.552959 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41b566-c709-43d4-99d9-ec84ef81c222 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:43.579954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41b566-c709-43d4-99d9-ec84ef81c222 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:43.585618 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7e6aa-7418-4a9e-8e67-8e0465ff34d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:43.586662 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7e6aa-7418-4a9e-8e67-8e0465ff34d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:43.587105 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7e6aa-7418-4a9e-8e67-8e0465ff34d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:43.587629 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7e6aa-7418-4a9e-8e67-8e0465ff34d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:43.589386 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7e6aa-7418-4a9e-8e67-8e0465ff34d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:43.592035 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7e6aa-7418-4a9e-8e67-8e0465ff34d5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:43.636462 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 938/1890] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:43 2022] POST /identity/v3/projects => generated 367 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:43.643278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42917781-d91b-4215-b0d0-5e6a14704397 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:43.660260 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42917781-d91b-4215-b0d0-5e6a14704397 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:43.706075 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:43.758613 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f2a43-513e-4a31-b50d-bd685042aecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:43.759343 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f2a43-513e-4a31-b50d-bd685042aecc admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:43.759624 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f2a43-513e-4a31-b50d-bd685042aecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:43.759906 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f2a43-513e-4a31-b50d-bd685042aecc admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:43.760776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f2a43-513e-4a31-b50d-bd685042aecc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:43.764190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f2a43-513e-4a31-b50d-bd685042aecc admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:43.797267 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 953/1891] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:43 2022] POST /identity/v3/users => generated 284 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:43.802676 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7e6aa-7418-4a9e-8e67-8e0465ff34d5 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:43.819210 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7e6aa-7418-4a9e-8e67-8e0465ff34d5 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:43.824205 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af45ed4-1e86-4c33-a939-053ae5c5240e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:43.825546 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af45ed4-1e86-4c33-a939-053ae5c5240e admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:43.826036 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af45ed4-1e86-4c33-a939-053ae5c5240e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:43.826467 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af45ed4-1e86-4c33-a939-053ae5c5240e admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:43.827236 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af45ed4-1e86-4c33-a939-053ae5c5240e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:43.830217 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af45ed4-1e86-4c33-a939-053ae5c5240e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:43.845044 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 939/1892] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:43.848216 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f2a43-513e-4a31-b50d-bd685042aecc admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:43.860273 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f2a43-513e-4a31-b50d-bd685042aecc admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:43.865499 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c551a161-4173-47c6-a1cd-b437003e2c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:43.866413 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551a161-4173-47c6-a1cd-b437003e2c40 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:43.866747 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551a161-4173-47c6-a1cd-b437003e2c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:43.867621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551a161-4173-47c6-a1cd-b437003e2c40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:43.868924 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c551a161-4173-47c6-a1cd-b437003e2c40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:43.870735 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c551a161-4173-47c6-a1cd-b437003e2c40 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:43.884244 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 954/1893] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:43.889859 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af45ed4-1e86-4c33-a939-053ae5c5240e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:43.901582 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af45ed4-1e86-4c33-a939-053ae5c5240e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:43.907035 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb1e3e-9a18-477a-853e-229c1ab1625f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:43.907990 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb1e3e-9a18-477a-853e-229c1ab1625f admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:43.908274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb1e3e-9a18-477a-853e-229c1ab1625f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:43.908550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb1e3e-9a18-477a-853e-229c1ab1625f admin admin] PATH_INFO: `/v3/projects/9610259c00014674a535b4d08bc20e26/users/8ea55a9a2ccc4eb892865ebce29fdd68/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:43.925771 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb1e3e-9a18-477a-853e-229c1ab1625f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9610259c00014674a535b4d08bc20e26, user_id=8ea55a9a2ccc4eb892865ebce29fdd68, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:43.928633 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb1e3e-9a18-477a-853e-229c1ab1625f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:43.945593 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 940/1894] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:43 2022] PUT /identity/v3/projects/9610259c00014674a535b4d08bc20e26/users/8ea55a9a2ccc4eb892865ebce29fdd68/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:12:43.958195 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acf8a5-e3b6-4672-a9a8-d46efea0f051 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:43.958664 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acf8a5-e3b6-4672-a9a8-d46efea0f051 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:43.958995 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acf8a5-e3b6-4672-a9a8-d46efea0f051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:43.990507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-b2acf8a5-e3b6-4672-a9a8-d46efea0f051 None None] MFA Rules not processed for user `8ea55a9a2ccc4eb892865ebce29fdd68`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:43.991771 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-b2acf8a5-e3b6-4672-a9a8-d46efea0f051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:44.069830 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 955/1895] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:43 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:44.074664 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb1e3e-9a18-477a-853e-229c1ab1625f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:44.149724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb1e3e-9a18-477a-853e-229c1ab1625f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:44.155079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba746dd-db09-4d32-821f-47c6945c29fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:44.155784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba746dd-db09-4d32-821f-47c6945c29fd admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:44.156067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba746dd-db09-4d32-821f-47c6945c29fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:44.156346 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba746dd-db09-4d32-821f-47c6945c29fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:44.157123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba746dd-db09-4d32-821f-47c6945c29fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:44.159096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba746dd-db09-4d32-821f-47c6945c29fd admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:44.191476 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 941/1896] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:44 2022] POST /identity/v3/projects => generated 367 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:44.196475 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acf8a5-e3b6-4672-a9a8-d46efea0f051 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:44.208548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acf8a5-e3b6-4672-a9a8-d46efea0f051 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:44.275311 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3cc8e-d6bc-491c-9b46-5b68a4233ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:44.276008 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3cc8e-d6bc-491c-9b46-5b68a4233ebc admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:44.276287 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3cc8e-d6bc-491c-9b46-5b68a4233ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:44.276567 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3cc8e-d6bc-491c-9b46-5b68a4233ebc admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:44.277269 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3cc8e-d6bc-491c-9b46-5b68a4233ebc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:44.279928 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3cc8e-d6bc-491c-9b46-5b68a4233ebc admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:44.307066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 956/1897] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:44 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:44.312034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba746dd-db09-4d32-821f-47c6945c29fd admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:44.324614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba746dd-db09-4d32-821f-47c6945c29fd admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:44.330879 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e1324-c14b-4e71-8120-d36082a027a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:44.331584 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e1324-c14b-4e71-8120-d36082a027a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:44.331944 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e1324-c14b-4e71-8120-d36082a027a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:44.332145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e1324-c14b-4e71-8120-d36082a027a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:44.332984 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e1324-c14b-4e71-8120-d36082a027a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:44.334972 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e1324-c14b-4e71-8120-d36082a027a9 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:44.350188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 942/1898] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:44.355173 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3cc8e-d6bc-491c-9b46-5b68a4233ebc admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:44.376268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3cc8e-d6bc-491c-9b46-5b68a4233ebc admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:44.388208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad4cfa-3ccf-4898-95fb-54331323492f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:44.389181 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad4cfa-3ccf-4898-95fb-54331323492f admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:44.389468 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad4cfa-3ccf-4898-95fb-54331323492f admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:44.389753 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad4cfa-3ccf-4898-95fb-54331323492f admin admin] PATH_INFO: `/v3/projects/a04432a31b874b939e284ccfedcc8a4d/users/1e5d08f8fff84078b46e84cd795a74b9/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:44.416324 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad4cfa-3ccf-4898-95fb-54331323492f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a04432a31b874b939e284ccfedcc8a4d, user_id=1e5d08f8fff84078b46e84cd795a74b9, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:44.420648 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad4cfa-3ccf-4898-95fb-54331323492f admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:44.433290 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:44.444479 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 957/1899] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:44 2022] PUT /identity/v3/projects/a04432a31b874b939e284ccfedcc8a4d/users/1e5d08f8fff84078b46e84cd795a74b9/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:44.447424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e1324-c14b-4e71-8120-d36082a027a9 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:44.476085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e1324-c14b-4e71-8120-d36082a027a9 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:44.515532 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:44.564318 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb80b0-5d97-418d-9724-4e1eb42f621d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:44.565697 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb80b0-5d97-418d-9724-4e1eb42f621d admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:44.565841 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb80b0-5d97-418d-9724-4e1eb42f621d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:44.566289 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb80b0-5d97-418d-9724-4e1eb42f621d admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:44.567595 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb80b0-5d97-418d-9724-4e1eb42f621d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:44.570241 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb80b0-5d97-418d-9724-4e1eb42f621d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:44.587965 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 943/1900] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:44.595318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad4cfa-3ccf-4898-95fb-54331323492f admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:44.611559 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad4cfa-3ccf-4898-95fb-54331323492f admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:44.626242 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516bd7fa-dfd9-4f77-bef3-00761758769b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:44.627322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516bd7fa-dfd9-4f77-bef3-00761758769b admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:44.627732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516bd7fa-dfd9-4f77-bef3-00761758769b admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:44.628154 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516bd7fa-dfd9-4f77-bef3-00761758769b admin admin] PATH_INFO: `/v3/projects/a04432a31b874b939e284ccfedcc8a4d/users/1e5d08f8fff84078b46e84cd795a74b9/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:44.630405 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516bd7fa-dfd9-4f77-bef3-00761758769b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a04432a31b874b939e284ccfedcc8a4d, user_id=1e5d08f8fff84078b46e84cd795a74b9, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:44.633575 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516bd7fa-dfd9-4f77-bef3-00761758769b admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:44.653220 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 958/1901] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:44 2022] PUT /identity/v3/projects/a04432a31b874b939e284ccfedcc8a4d/users/1e5d08f8fff84078b46e84cd795a74b9/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:12:44.670008 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbe7de-3a61-47c4-a91e-33133a8cbe6a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:44.670284 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbe7de-3a61-47c4-a91e-33133a8cbe6a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:44.670714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbe7de-3a61-47c4-a91e-33133a8cbe6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:44.709508 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-2bcbe7de-3a61-47c4-a91e-33133a8cbe6a None None] MFA Rules not processed for user `1e5d08f8fff84078b46e84cd795a74b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:44.713567 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2bcbe7de-3a61-47c4-a91e-33133a8cbe6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:44.841613 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 944/1902] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:44 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:12:44.853333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516bd7fa-dfd9-4f77-bef3-00761758769b admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:44.855299 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-516bd7fa-dfd9-4f77-bef3-00761758769b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:44.972327 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516bd7fa-dfd9-4f77-bef3-00761758769b admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:44.978603 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb5421-6ee8-44de-a00e-4a554735284f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:44.979791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb5421-6ee8-44de-a00e-4a554735284f service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:44.980297 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb5421-6ee8-44de-a00e-4a554735284f service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:44.981220 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb5421-6ee8-44de-a00e-4a554735284f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:45.001822 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cb5421-6ee8-44de-a00e-4a554735284f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:45.004291 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cb5421-6ee8-44de-a00e-4a554735284f service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:45.026011 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 959/1903] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:44 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:12:45.145368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbe7de-3a61-47c4-a91e-33133a8cbe6a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:45.156624 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbe7de-3a61-47c4-a91e-33133a8cbe6a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:45.161135 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc3239e-6cec-407e-9cf2-6dc2e3b09300 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:45.161904 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3239e-6cec-407e-9cf2-6dc2e3b09300 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:45.162127 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3239e-6cec-407e-9cf2-6dc2e3b09300 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:45.162405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3239e-6cec-407e-9cf2-6dc2e3b09300 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:45.270219 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc3239e-6cec-407e-9cf2-6dc2e3b09300 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:45.272656 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc3239e-6cec-407e-9cf2-6dc2e3b09300 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:45.293692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 945/1904] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:45 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:49.785809 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb5421-6ee8-44de-a00e-4a554735284f service cinder] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:49.788718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-64cb5421-6ee8-44de-a00e-4a554735284f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:49.963658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb5421-6ee8-44de-a00e-4a554735284f service cinder] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:49.975778 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e494822-2456-4644-aafe-7433a87e72df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:49.977964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e494822-2456-4644-aafe-7433a87e72df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:49.978480 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e494822-2456-4644-aafe-7433a87e72df admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:49.978969 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e494822-2456-4644-aafe-7433a87e72df admin admin] PATH_INFO: `/v3/users/27e845541c424176babffa617fa09a6a` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:49.983044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e494822-2456-4644-aafe-7433a87e72df admin admin] RBAC: Authorizing `identity:delete_user(user_id=27e845541c424176babffa617fa09a6a)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:49.987303 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e494822-2456-4644-aafe-7433a87e72df admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:50.102069 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6e494822-2456-4644-aafe-7433a87e72df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27e845541c424176babffa617fa09a6a'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:50.124238 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6e494822-2456-4644-aafe-7433a87e72df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27e845541c424176babffa617fa09a6a'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:50.166573 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6e494822-2456-4644-aafe-7433a87e72df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27e845541c424176babffa617fa09a6a'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:50.167276 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6e494822-2456-4644-aafe-7433a87e72df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27e845541c424176babffa617fa09a6a'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:50.180367 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6e494822-2456-4644-aafe-7433a87e72df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27e845541c424176babffa617fa09a6a'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:50.193525 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 960/1905] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:49 2022] DELETE /identity/v3/users/27e845541c424176babffa617fa09a6a => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:50.200406 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b3e08-4d0e-44a5-b41d-06e0bc6a8e67 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:50.201081 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b3e08-4d0e-44a5-b41d-06e0bc6a8e67 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:50.201283 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b3e08-4d0e-44a5-b41d-06e0bc6a8e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:50.234559 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9c1b3e08-4d0e-44a5-b41d-06e0bc6a8e67 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:50.236869 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9c1b3e08-4d0e-44a5-b41d-06e0bc6a8e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:50.397275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:50.451142 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 946/1906] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:12:50.576241 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e494822-2456-4644-aafe-7433a87e72df admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:50.578794 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-6e494822-2456-4644-aafe-7433a87e72df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:50.605403 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:50.740213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e494822-2456-4644-aafe-7433a87e72df admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:50.762874 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b9c26-2a71-4779-8417-ede46aef3869 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:50.764073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b9c26-2a71-4779-8417-ede46aef3869 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:50.764911 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b9c26-2a71-4779-8417-ede46aef3869 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:50.765090 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b9c26-2a71-4779-8417-ede46aef3869 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:50.791140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b9c26-2a71-4779-8417-ede46aef3869 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:50.795070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b9c26-2a71-4779-8417-ede46aef3869 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:50.821297 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 961/1907] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:51.136686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b3e08-4d0e-44a5-b41d-06e0bc6a8e67 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:51.139976 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9c1b3e08-4d0e-44a5-b41d-06e0bc6a8e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:51.169330 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b3e08-4d0e-44a5-b41d-06e0bc6a8e67 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:51.178945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c21133-53e3-4968-a1a7-add53ef7a86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:51.180830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c21133-53e3-4968-a1a7-add53ef7a86e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:51.181644 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c21133-53e3-4968-a1a7-add53ef7a86e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:51.185393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c21133-53e3-4968-a1a7-add53ef7a86e admin admin] PATH_INFO: `/v3/users/c265e168184342138a0909202440c560` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:51.188345 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c21133-53e3-4968-a1a7-add53ef7a86e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c265e168184342138a0909202440c560)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:51.193680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c21133-53e3-4968-a1a7-add53ef7a86e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:51.392155 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-47c21133-53e3-4968-a1a7-add53ef7a86e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c265e168184342138a0909202440c560'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:51.453698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-47c21133-53e3-4968-a1a7-add53ef7a86e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c265e168184342138a0909202440c560'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:51.482640 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-47c21133-53e3-4968-a1a7-add53ef7a86e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c265e168184342138a0909202440c560'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:51.514875 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-47c21133-53e3-4968-a1a7-add53ef7a86e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c265e168184342138a0909202440c560'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:51.531024 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-47c21133-53e3-4968-a1a7-add53ef7a86e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c265e168184342138a0909202440c560'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:51.568899 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f31b47-4ffb-40c8-a039-be7751b2a704 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:51.569272 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f31b47-4ffb-40c8-a039-be7751b2a704 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:51.569556 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f31b47-4ffb-40c8-a039-be7751b2a704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:51.570048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 947/1908] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:51 2022] DELETE /identity/v3/users/c265e168184342138a0909202440c560 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:51.618367 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-f7f31b47-4ffb-40c8-a039-be7751b2a704 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:51.619533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-f7f31b47-4ffb-40c8-a039-be7751b2a704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:51.640024 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c21133-53e3-4968-a1a7-add53ef7a86e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:51.641575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-47c21133-53e3-4968-a1a7-add53ef7a86e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:51.830616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 962/1909] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:51.846954 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f31b47-4ffb-40c8-a039-be7751b2a704 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:51.849229 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-f7f31b47-4ffb-40c8-a039-be7751b2a704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:51.879056 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c21133-53e3-4968-a1a7-add53ef7a86e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:51.887026 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acab6c0-be81-46fe-95f2-529e30db7053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:51.888508 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acab6c0-be81-46fe-95f2-529e30db7053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:51.888947 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acab6c0-be81-46fe-95f2-529e30db7053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:51.889329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acab6c0-be81-46fe-95f2-529e30db7053 admin admin] PATH_INFO: `/v3/projects/3ccdb67ec1de46ce91b788aefa3f2182` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:51.891389 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acab6c0-be81-46fe-95f2-529e30db7053 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ccdb67ec1de46ce91b788aefa3f2182)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:51.895049 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acab6c0-be81-46fe-95f2-529e30db7053 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:51.929107 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f31b47-4ffb-40c8-a039-be7751b2a704 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:51.936946 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c95a2-76f7-41c5-9b49-f0e74c7f96ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:51.937739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c95a2-76f7-41c5-9b49-f0e74c7f96ae service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:51.938123 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c95a2-76f7-41c5-9b49-f0e74c7f96ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:51.938516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c95a2-76f7-41c5-9b49-f0e74c7f96ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:51.958069 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3c95a2-76f7-41c5-9b49-f0e74c7f96ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:51.962153 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3c95a2-76f7-41c5-9b49-f0e74c7f96ae service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:51.985622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 963/1910] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:12:52.001030 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5acab6c0-be81-46fe-95f2-529e30db7053 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ccdb67ec1de46ce91b788aefa3f2182', 'request_id': 'req-5acab6c0-be81-46fe-95f2-529e30db7053'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:52.024015 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5acab6c0-be81-46fe-95f2-529e30db7053 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ccdb67ec1de46ce91b788aefa3f2182', 'request_id': 'req-5acab6c0-be81-46fe-95f2-529e30db7053'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:52.030496 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:52.044739 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5acab6c0-be81-46fe-95f2-529e30db7053 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ccdb67ec1de46ce91b788aefa3f2182', 'request_id': 'req-5acab6c0-be81-46fe-95f2-529e30db7053'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:52.072253 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5acab6c0-be81-46fe-95f2-529e30db7053 admin admin] The token cache is being invalidate because project 3ccdb67ec1de46ce91b788aefa3f2182 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:52.073316 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5acab6c0-be81-46fe-95f2-529e30db7053 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:52.074940 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5acab6c0-be81-46fe-95f2-529e30db7053 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:52.083251 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 948/1911] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:51 2022] DELETE /identity/v3/projects/3ccdb67ec1de46ce91b788aefa3f2182 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:52.087229 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c95a2-76f7-41c5-9b49-f0e74c7f96ae service neutron] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:52.114017 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c95a2-76f7-41c5-9b49-f0e74c7f96ae service neutron] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:52.218073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:52.232557 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1159356-dddc-4c92-9844-846f899555d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:52.234039 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1159356-dddc-4c92-9844-846f899555d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:52.234245 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1159356-dddc-4c92-9844-846f899555d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:52.234618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1159356-dddc-4c92-9844-846f899555d3 admin admin] PATH_INFO: `/v3/users/bec56597817f4caa9f8a81d5cf844eac` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:52.236451 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1159356-dddc-4c92-9844-846f899555d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bec56597817f4caa9f8a81d5cf844eac)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:52.239765 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1159356-dddc-4c92-9844-846f899555d3 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:52.331242 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-c1159356-dddc-4c92-9844-846f899555d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bec56597817f4caa9f8a81d5cf844eac'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:52.343169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-c1159356-dddc-4c92-9844-846f899555d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bec56597817f4caa9f8a81d5cf844eac'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:52.365182 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-c1159356-dddc-4c92-9844-846f899555d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bec56597817f4caa9f8a81d5cf844eac'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:52.365182 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-c1159356-dddc-4c92-9844-846f899555d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bec56597817f4caa9f8a81d5cf844eac'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:52.378079 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-c1159356-dddc-4c92-9844-846f899555d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bec56597817f4caa9f8a81d5cf844eac'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:52.393741 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 964/1912] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:52 2022] DELETE /identity/v3/users/bec56597817f4caa9f8a81d5cf844eac => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:52.528737 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acab6c0-be81-46fe-95f2-529e30db7053 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:52.531856 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5acab6c0-be81-46fe-95f2-529e30db7053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:52.566693 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acab6c0-be81-46fe-95f2-529e30db7053 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:52.728322 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c8c74-46e8-4441-8f7a-3a8b9c027d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:52.729742 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c8c74-46e8-4441-8f7a-3a8b9c027d04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:52.730817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c8c74-46e8-4441-8f7a-3a8b9c027d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:52.731233 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c8c74-46e8-4441-8f7a-3a8b9c027d04 admin admin] PATH_INFO: `/v3/projects/85ff1f13e634456d92e39796a86e9419` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:52.732948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069c8c74-46e8-4441-8f7a-3a8b9c027d04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85ff1f13e634456d92e39796a86e9419)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:52.735777 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069c8c74-46e8-4441-8f7a-3a8b9c027d04 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:52.829408 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-069c8c74-46e8-4441-8f7a-3a8b9c027d04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85ff1f13e634456d92e39796a86e9419', 'request_id': 'req-069c8c74-46e8-4441-8f7a-3a8b9c027d04'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:52.839139 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-069c8c74-46e8-4441-8f7a-3a8b9c027d04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85ff1f13e634456d92e39796a86e9419', 'request_id': 'req-069c8c74-46e8-4441-8f7a-3a8b9c027d04'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:52.851833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-069c8c74-46e8-4441-8f7a-3a8b9c027d04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85ff1f13e634456d92e39796a86e9419', 'request_id': 'req-069c8c74-46e8-4441-8f7a-3a8b9c027d04'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:52.886491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-069c8c74-46e8-4441-8f7a-3a8b9c027d04 admin admin] The token cache is being invalidate because project 85ff1f13e634456d92e39796a86e9419 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:52.887553 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-069c8c74-46e8-4441-8f7a-3a8b9c027d04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:52.890455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-069c8c74-46e8-4441-8f7a-3a8b9c027d04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:52.899215 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 949/1913] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:52 2022] DELETE /identity/v3/projects/85ff1f13e634456d92e39796a86e9419 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:52.903830 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1159356-dddc-4c92-9844-846f899555d3 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:52.905509 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-c1159356-dddc-4c92-9844-846f899555d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:52.928743 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c8c74-46e8-4441-8f7a-3a8b9c027d04 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:52.930298 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-069c8c74-46e8-4441-8f7a-3a8b9c027d04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:52.932028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1159356-dddc-4c92-9844-846f899555d3 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:53.040281 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c8c74-46e8-4441-8f7a-3a8b9c027d04 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:53.046110 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc5e542-edb3-4404-8221-500bb9c20980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:53.047072 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5e542-edb3-4404-8221-500bb9c20980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:53.047514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5e542-edb3-4404-8221-500bb9c20980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:53.047839 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5e542-edb3-4404-8221-500bb9c20980 admin admin] PATH_INFO: `/v3/projects/5abb815771094d5c9b2f3833298c5b9a` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:53.049134 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c09726-db54-4905-b7c4-8b71e0508bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:53.049428 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc5e542-edb3-4404-8221-500bb9c20980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5abb815771094d5c9b2f3833298c5b9a)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:53.050254 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c09726-db54-4905-b7c4-8b71e0508bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:53.050638 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c09726-db54-4905-b7c4-8b71e0508bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:53.050992 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c09726-db54-4905-b7c4-8b71e0508bff admin admin] PATH_INFO: `/v3/users/e68bec05f04049a8976c63d17a711900` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:53.051684 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc5e542-edb3-4404-8221-500bb9c20980 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:53.052364 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c09726-db54-4905-b7c4-8b71e0508bff admin admin] RBAC: Authorizing `identity:delete_user(user_id=e68bec05f04049a8976c63d17a711900)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:53.054478 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c09726-db54-4905-b7c4-8b71e0508bff admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:53.086043 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:53.119439 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-89c09726-db54-4905-b7c4-8b71e0508bff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e68bec05f04049a8976c63d17a711900'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:53.123275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fcc5e542-edb3-4404-8221-500bb9c20980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5abb815771094d5c9b2f3833298c5b9a', 'request_id': 'req-fcc5e542-edb3-4404-8221-500bb9c20980'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:53.133707 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fcc5e542-edb3-4404-8221-500bb9c20980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5abb815771094d5c9b2f3833298c5b9a', 'request_id': 'req-fcc5e542-edb3-4404-8221-500bb9c20980'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:53.134818 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-89c09726-db54-4905-b7c4-8b71e0508bff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e68bec05f04049a8976c63d17a711900'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:53.145438 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fcc5e542-edb3-4404-8221-500bb9c20980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5abb815771094d5c9b2f3833298c5b9a', 'request_id': 'req-fcc5e542-edb3-4404-8221-500bb9c20980'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:53.156203 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-89c09726-db54-4905-b7c4-8b71e0508bff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e68bec05f04049a8976c63d17a711900'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:53.157440 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-89c09726-db54-4905-b7c4-8b71e0508bff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e68bec05f04049a8976c63d17a711900'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:53.161064 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:53.161191 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fcc5e542-edb3-4404-8221-500bb9c20980 admin admin] The token cache is being invalidate because project 5abb815771094d5c9b2f3833298c5b9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:53.161358 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fcc5e542-edb3-4404-8221-500bb9c20980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:53.162139 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fcc5e542-edb3-4404-8221-500bb9c20980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:53.164588 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 950/1914] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:52 2022] DELETE /identity/v3/projects/5abb815771094d5c9b2f3833298c5b9a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:53.170882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-89c09726-db54-4905-b7c4-8b71e0508bff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e68bec05f04049a8976c63d17a711900'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:53.172340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91966922-86b6-42f8-8c01-a992f66e2957 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:53.173171 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91966922-86b6-42f8-8c01-a992f66e2957 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:53.173304 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91966922-86b6-42f8-8c01-a992f66e2957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:53.179410 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 965/1915] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:52 2022] DELETE /identity/v3/users/e68bec05f04049a8976c63d17a711900 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:12:53.197341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-91966922-86b6-42f8-8c01-a992f66e2957 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:53.198701 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-91966922-86b6-42f8-8c01-a992f66e2957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:53.284992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 951/1916] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:53.293820 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d202f0-adb8-467b-9153-d798b55312e4 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:53.294126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d202f0-adb8-467b-9153-d798b55312e4 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:53.294483 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d202f0-adb8-467b-9153-d798b55312e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:53.316616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-a2d202f0-adb8-467b-9153-d798b55312e4 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:53.317954 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-a2d202f0-adb8-467b-9153-d798b55312e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:53.329190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 966/1917] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:53.333812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91966922-86b6-42f8-8c01-a992f66e2957 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:53.347456 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91966922-86b6-42f8-8c01-a992f66e2957 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:53.351214 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0ab0b-3c41-4751-bdbd-1fca74031680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:53.351998 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0ab0b-3c41-4751-bdbd-1fca74031680 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:53.352239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0ab0b-3c41-4751-bdbd-1fca74031680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:53.352596 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0ab0b-3c41-4751-bdbd-1fca74031680 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:53.353639 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f0ab0b-3c41-4751-bdbd-1fca74031680 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:12:53.354197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f0ab0b-3c41-4751-bdbd-1fca74031680 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:53.356392 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f0ab0b-3c41-4751-bdbd-1fca74031680 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:53.386816 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 952/1918] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:12:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:12:53.393097 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d202f0-adb8-467b-9153-d798b55312e4 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:53.403151 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d202f0-adb8-467b-9153-d798b55312e4 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:53.408861 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6926763-bd40-45b2-9bd1-fc797a8ae810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:53.409753 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6926763-bd40-45b2-9bd1-fc797a8ae810 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:53.410029 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6926763-bd40-45b2-9bd1-fc797a8ae810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:53.410399 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6926763-bd40-45b2-9bd1-fc797a8ae810 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:53.411056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6926763-bd40-45b2-9bd1-fc797a8ae810 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:53.413021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6926763-bd40-45b2-9bd1-fc797a8ae810 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:53.450659 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 967/1919] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:53 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:53.451925 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0ab0b-3c41-4751-bdbd-1fca74031680 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:53.470525 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0ab0b-3c41-4751-bdbd-1fca74031680 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:53.594348 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f114a4-bdef-45dd-a493-cdcdd0fefda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:53.595617 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f114a4-bdef-45dd-a493-cdcdd0fefda0 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:53.596057 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f114a4-bdef-45dd-a493-cdcdd0fefda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:53.596741 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f114a4-bdef-45dd-a493-cdcdd0fefda0 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:53.597802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f114a4-bdef-45dd-a493-cdcdd0fefda0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:53.601055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f114a4-bdef-45dd-a493-cdcdd0fefda0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:53.606708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6926763-bd40-45b2-9bd1-fc797a8ae810 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:53.609982 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-d6926763-bd40-45b2-9bd1-fc797a8ae810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:53.646046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 953/1920] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:53 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:53.648038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f114a4-bdef-45dd-a493-cdcdd0fefda0 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:53.653968 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6926763-bd40-45b2-9bd1-fc797a8ae810 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:53.661246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f114a4-bdef-45dd-a493-cdcdd0fefda0 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:53.666473 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc42c6-3117-4abe-8d53-de2e4fa0312d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:53.666702 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbbacc-367f-4a05-99bb-357084eec802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:53.667149 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbbacc-367f-4a05-99bb-357084eec802 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:53.667759 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbbacc-367f-4a05-99bb-357084eec802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:53.667759 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbbacc-367f-4a05-99bb-357084eec802 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:53.669034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc42c6-3117-4abe-8d53-de2e4fa0312d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:53.669505 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc42c6-3117-4abe-8d53-de2e4fa0312d admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:53.670064 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cbbacc-367f-4a05-99bb-357084eec802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:53.670227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc42c6-3117-4abe-8d53-de2e4fa0312d admin admin] PATH_INFO: `/v3/projects/3dc7ab8276a944fab6ab89374628d2b3` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:53.671702 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cbbacc-367f-4a05-99bb-357084eec802 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:53.672907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc42c6-3117-4abe-8d53-de2e4fa0312d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dc7ab8276a944fab6ab89374628d2b3)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:53.676334 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc42c6-3117-4abe-8d53-de2e4fa0312d admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:53.684627 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 954/1921] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:12:53.689188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbbacc-367f-4a05-99bb-357084eec802 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:53.705012 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbbacc-367f-4a05-99bb-357084eec802 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:53.711179 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0074d1-30c6-46b0-b61e-ba8fee854601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:53.711967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0074d1-30c6-46b0-b61e-ba8fee854601 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:53.712332 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0074d1-30c6-46b0-b61e-ba8fee854601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:53.712641 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0074d1-30c6-46b0-b61e-ba8fee854601 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:53.713584 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0074d1-30c6-46b0-b61e-ba8fee854601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:53.716044 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0074d1-30c6-46b0-b61e-ba8fee854601 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:53.728753 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 955/1922] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:53.735609 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0074d1-30c6-46b0-b61e-ba8fee854601 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:53.750737 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0074d1-30c6-46b0-b61e-ba8fee854601 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:53.755036 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:53.755506 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-3ddc42c6-3117-4abe-8d53-de2e4fa0312d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dc7ab8276a944fab6ab89374628d2b3', 'request_id': 'req-3ddc42c6-3117-4abe-8d53-de2e4fa0312d'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:53.766998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-3ddc42c6-3117-4abe-8d53-de2e4fa0312d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dc7ab8276a944fab6ab89374628d2b3', 'request_id': 'req-3ddc42c6-3117-4abe-8d53-de2e4fa0312d'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:53.791360 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-3ddc42c6-3117-4abe-8d53-de2e4fa0312d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dc7ab8276a944fab6ab89374628d2b3', 'request_id': 'req-3ddc42c6-3117-4abe-8d53-de2e4fa0312d'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:53.808362 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-3ddc42c6-3117-4abe-8d53-de2e4fa0312d admin admin] The token cache is being invalidate because project 3dc7ab8276a944fab6ab89374628d2b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:53.809272 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-3ddc42c6-3117-4abe-8d53-de2e4fa0312d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:53.809272 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-3ddc42c6-3117-4abe-8d53-de2e4fa0312d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:53.816606 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 968/1923] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:53 2022] DELETE /identity/v3/projects/3dc7ab8276a944fab6ab89374628d2b3 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:53.818031 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd78089-8060-4fff-ac4f-72d223591955 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:53.818324 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd78089-8060-4fff-ac4f-72d223591955 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:53.818794 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd78089-8060-4fff-ac4f-72d223591955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:53.837867 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-7bd78089-8060-4fff-ac4f-72d223591955 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:53.839132 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-7bd78089-8060-4fff-ac4f-72d223591955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:53.873855 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65e5f5-b894-4134-ba3d-ecc80d208c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:53.875440 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65e5f5-b894-4134-ba3d-ecc80d208c3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:53.875719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65e5f5-b894-4134-ba3d-ecc80d208c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:53.876290 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65e5f5-b894-4134-ba3d-ecc80d208c3f admin admin] PATH_INFO: `/v3/projects/96e729e07876487bac9e8016467350a3/users/d8b5bac3d35c414eaaa10b6a91d8d91d/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:53.899160 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc65e5f5-b894-4134-ba3d-ecc80d208c3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=96e729e07876487bac9e8016467350a3, user_id=d8b5bac3d35c414eaaa10b6a91d8d91d, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:53.901981 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc65e5f5-b894-4134-ba3d-ecc80d208c3f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:53.920996 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 956/1924] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:53 2022] PUT /identity/v3/projects/96e729e07876487bac9e8016467350a3/users/d8b5bac3d35c414eaaa10b6a91d8d91d/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:12:53.922677 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 969/1925] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:53.933174 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78a75e-a129-407a-8cec-0a97833359b8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:53.934083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78a75e-a129-407a-8cec-0a97833359b8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:53.934808 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d9520-21df-4f3e-92d3-0627c95cae24 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:53.934910 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78a75e-a129-407a-8cec-0a97833359b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:53.935012 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d9520-21df-4f3e-92d3-0627c95cae24 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:53.935771 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d9520-21df-4f3e-92d3-0627c95cae24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:53.968255 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-3c1d9520-21df-4f3e-92d3-0627c95cae24 None None] MFA Rules not processed for user `d8b5bac3d35c414eaaa10b6a91d8d91d`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:53.968597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8c78a75e-a129-407a-8cec-0a97833359b8 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:53.969477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-3c1d9520-21df-4f3e-92d3-0627c95cae24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:53.970893 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8c78a75e-a129-407a-8cec-0a97833359b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:54.021323 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:54.057570 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 970/1926] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:54.075995 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d9520-21df-4f3e-92d3-0627c95cae24 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:54.076891 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 957/1927] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:12:54.077641 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-3c1d9520-21df-4f3e-92d3-0627c95cae24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:54.079815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78a75e-a129-407a-8cec-0a97833359b8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:54.099138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78a75e-a129-407a-8cec-0a97833359b8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:54.111978 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af89c6-793f-4fd3-a31b-6f1ccefc013e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:54.113870 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af89c6-793f-4fd3-a31b-6f1ccefc013e admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:54.114399 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af89c6-793f-4fd3-a31b-6f1ccefc013e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:54.114881 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af89c6-793f-4fd3-a31b-6f1ccefc013e admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:54.116128 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af89c6-793f-4fd3-a31b-6f1ccefc013e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:12:54.116670 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af89c6-793f-4fd3-a31b-6f1ccefc013e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:54.119827 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af89c6-793f-4fd3-a31b-6f1ccefc013e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:54.142137 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 958/1928] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:12:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:12:54.143267 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af89c6-793f-4fd3-a31b-6f1ccefc013e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:54.154629 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af89c6-793f-4fd3-a31b-6f1ccefc013e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:54.161016 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7504313-fecb-4502-82ee-7e651ed66440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:54.161680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7504313-fecb-4502-82ee-7e651ed66440 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:54.161978 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7504313-fecb-4502-82ee-7e651ed66440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:54.162259 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7504313-fecb-4502-82ee-7e651ed66440 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:54.162957 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7504313-fecb-4502-82ee-7e651ed66440 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:54.164934 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7504313-fecb-4502-82ee-7e651ed66440 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:54.165691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d9520-21df-4f3e-92d3-0627c95cae24 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:54.172341 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66386c72-ab89-42b6-a138-4c7bdf930c02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:54.173041 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66386c72-ab89-42b6-a138-4c7bdf930c02 service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:54.173495 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66386c72-ab89-42b6-a138-4c7bdf930c02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:54.174612 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66386c72-ab89-42b6-a138-4c7bdf930c02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:54.186645 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66386c72-ab89-42b6-a138-4c7bdf930c02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:54.188300 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66386c72-ab89-42b6-a138-4c7bdf930c02 service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:54.200445 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 971/1929] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:54.211210 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 959/1930] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:54 2022] POST /identity/v3/projects => generated 387 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:54.216504 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66386c72-ab89-42b6-a138-4c7bdf930c02 service cinder] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:54.231282 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66386c72-ab89-42b6-a138-4c7bdf930c02 service cinder] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:54.297792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f541e-d18e-49c8-9c64-bbb8cb737a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:54.298520 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f541e-d18e-49c8-9c64-bbb8cb737a30 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:54.298849 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f541e-d18e-49c8-9c64-bbb8cb737a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:54.299132 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f541e-d18e-49c8-9c64-bbb8cb737a30 admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:54.299876 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f541e-d18e-49c8-9c64-bbb8cb737a30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:54.301838 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f541e-d18e-49c8-9c64-bbb8cb737a30 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:54.331135 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 972/1931] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:54 2022] POST /identity/v3/users => generated 294 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:12:54.332298 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7504313-fecb-4502-82ee-7e651ed66440 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:54.344849 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7504313-fecb-4502-82ee-7e651ed66440 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:54.350679 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f788dee-85e5-4090-97be-2cdb0e2aa1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:54.351378 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f788dee-85e5-4090-97be-2cdb0e2aa1aa admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:54.351667 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f788dee-85e5-4090-97be-2cdb0e2aa1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:54.351945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f788dee-85e5-4090-97be-2cdb0e2aa1aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:54.352682 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f788dee-85e5-4090-97be-2cdb0e2aa1aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:54.354375 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f788dee-85e5-4090-97be-2cdb0e2aa1aa admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:54.368433 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 960/1932] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:12:54.374128 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f541e-d18e-49c8-9c64-bbb8cb737a30 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:54.384903 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f541e-d18e-49c8-9c64-bbb8cb737a30 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:54.391076 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7c3b9-a58a-42d4-858f-f29651242a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:54.391825 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7c3b9-a58a-42d4-858f-f29651242a5c admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:54.392112 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7c3b9-a58a-42d4-858f-f29651242a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:54.392389 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7c3b9-a58a-42d4-858f-f29651242a5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:54.393302 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f7c3b9-a58a-42d4-858f-f29651242a5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:54.395159 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f7c3b9-a58a-42d4-858f-f29651242a5c admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:54.410397 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 973/1933] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:54.416397 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f788dee-85e5-4090-97be-2cdb0e2aa1aa admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:54.433020 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f788dee-85e5-4090-97be-2cdb0e2aa1aa admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:54.441945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea29fbf-2878-4b65-8fe8-e5e3e8662832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:54.443481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea29fbf-2878-4b65-8fe8-e5e3e8662832 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:54.443945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea29fbf-2878-4b65-8fe8-e5e3e8662832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:54.444475 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea29fbf-2878-4b65-8fe8-e5e3e8662832 admin admin] PATH_INFO: `/v3/projects/f2afb72b5a6f4246951d5050c4a56ff0/users/230f54141c364d498ee439efa2cebe16/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:54.470126 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea29fbf-2878-4b65-8fe8-e5e3e8662832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2afb72b5a6f4246951d5050c4a56ff0, user_id=230f54141c364d498ee439efa2cebe16, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:54.473973 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea29fbf-2878-4b65-8fe8-e5e3e8662832 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:54.488570 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 961/1934] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:54 2022] PUT /identity/v3/projects/f2afb72b5a6f4246951d5050c4a56ff0/users/230f54141c364d498ee439efa2cebe16/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:54.500948 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41960e16-997d-46e3-a0f7-839a1b9bc130 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:54.500948 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41960e16-997d-46e3-a0f7-839a1b9bc130 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:54.501253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41960e16-997d-46e3-a0f7-839a1b9bc130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:54.566608 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-41960e16-997d-46e3-a0f7-839a1b9bc130 None None] MFA Rules not processed for user `230f54141c364d498ee439efa2cebe16`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:54.569183 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-41960e16-997d-46e3-a0f7-839a1b9bc130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:54.657366 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:54.685263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 974/1935] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:12:54.690592 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea29fbf-2878-4b65-8fe8-e5e3e8662832 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:54.713760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:54.786794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea29fbf-2878-4b65-8fe8-e5e3e8662832 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:54.793751 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2485ea-cdb9-43fc-a957-16dde0c04e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:54.794714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2485ea-cdb9-43fc-a957-16dde0c04e64 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:54.794995 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2485ea-cdb9-43fc-a957-16dde0c04e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:54.795273 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2485ea-cdb9-43fc-a957-16dde0c04e64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:54.796207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2485ea-cdb9-43fc-a957-16dde0c04e64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:54.798611 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2485ea-cdb9-43fc-a957-16dde0c04e64 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:54.840653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 962/1936] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:54 2022] POST /identity/v3/projects => generated 387 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:12:54.842690 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41960e16-997d-46e3-a0f7-839a1b9bc130 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:54.863928 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41960e16-997d-46e3-a0f7-839a1b9bc130 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:54.947695 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ec0a2-a24e-499d-92fc-9a3a0fa82064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:54.948727 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ec0a2-a24e-499d-92fc-9a3a0fa82064 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:54.948727 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ec0a2-a24e-499d-92fc-9a3a0fa82064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:54.949092 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ec0a2-a24e-499d-92fc-9a3a0fa82064 admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:54.949659 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ec0a2-a24e-499d-92fc-9a3a0fa82064 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:54.952219 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ec0a2-a24e-499d-92fc-9a3a0fa82064 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:54.988806 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 975/1937] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:54 2022] POST /identity/v3/users => generated 300 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:12:54.991086 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2485ea-cdb9-43fc-a957-16dde0c04e64 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:55.006314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2485ea-cdb9-43fc-a957-16dde0c04e64 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:55.012544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dde48d-7f8c-4702-8c15-a38ed8be7315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:55.013334 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dde48d-7f8c-4702-8c15-a38ed8be7315 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:55.013622 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dde48d-7f8c-4702-8c15-a38ed8be7315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:55.013951 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dde48d-7f8c-4702-8c15-a38ed8be7315 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:55.014718 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dde48d-7f8c-4702-8c15-a38ed8be7315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:55.016367 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dde48d-7f8c-4702-8c15-a38ed8be7315 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:55.029559 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 963/1938] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:12:55.035168 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ec0a2-a24e-499d-92fc-9a3a0fa82064 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:55.047722 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ec0a2-a24e-499d-92fc-9a3a0fa82064 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:55.052874 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f911b0-6fe4-4e02-b6f0-47d24fc7f5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:55.053787 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f911b0-6fe4-4e02-b6f0-47d24fc7f5be admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:55.054184 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f911b0-6fe4-4e02-b6f0-47d24fc7f5be admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:55.054462 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f911b0-6fe4-4e02-b6f0-47d24fc7f5be admin admin] PATH_INFO: `/v3/projects/964b13e396a94991a2a48314b46b3298/users/14be012b806d43539bee74e540fe8148/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:55.077994 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f911b0-6fe4-4e02-b6f0-47d24fc7f5be admin admin] RBAC: Authorizing `identity:create_grant(project_id=964b13e396a94991a2a48314b46b3298, user_id=14be012b806d43539bee74e540fe8148, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:55.082744 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f911b0-6fe4-4e02-b6f0-47d24fc7f5be admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:55.098785 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 976/1939] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:55 2022] PUT /identity/v3/projects/964b13e396a94991a2a48314b46b3298/users/14be012b806d43539bee74e540fe8148/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:55.107332 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dde48d-7f8c-4702-8c15-a38ed8be7315 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:55.118522 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dde48d-7f8c-4702-8c15-a38ed8be7315 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:55.213852 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c6afa-160e-493e-842e-7b155a4a0f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:55.214739 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c6afa-160e-493e-842e-7b155a4a0f54 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:55.215166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c6afa-160e-493e-842e-7b155a4a0f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:55.215589 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c6afa-160e-493e-842e-7b155a4a0f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:55.216739 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75c6afa-160e-493e-842e-7b155a4a0f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:55.219046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75c6afa-160e-493e-842e-7b155a4a0f54 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:55.234638 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 964/1940] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:55.239707 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f911b0-6fe4-4e02-b6f0-47d24fc7f5be admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:55.260957 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f911b0-6fe4-4e02-b6f0-47d24fc7f5be admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:55.274645 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95490877-3a1a-4e07-9aad-7ef511da7e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:55.275574 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95490877-3a1a-4e07-9aad-7ef511da7e03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:55.275984 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95490877-3a1a-4e07-9aad-7ef511da7e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:55.276675 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95490877-3a1a-4e07-9aad-7ef511da7e03 admin admin] PATH_INFO: `/v3/projects/964b13e396a94991a2a48314b46b3298/users/14be012b806d43539bee74e540fe8148/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:55.279294 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95490877-3a1a-4e07-9aad-7ef511da7e03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=964b13e396a94991a2a48314b46b3298, user_id=14be012b806d43539bee74e540fe8148, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:55.282112 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95490877-3a1a-4e07-9aad-7ef511da7e03 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:55.295249 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 977/1941] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:55 2022] PUT /identity/v3/projects/964b13e396a94991a2a48314b46b3298/users/14be012b806d43539bee74e540fe8148/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:55.305766 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969ddad-230a-4431-b100-99d118409b87 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:55.306465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969ddad-230a-4431-b100-99d118409b87 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:55.306779 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969ddad-230a-4431-b100-99d118409b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:55.340766 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e969ddad-230a-4431-b100-99d118409b87 None None] MFA Rules not processed for user `14be012b806d43539bee74e540fe8148`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:55.342905 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e969ddad-230a-4431-b100-99d118409b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:55.444682 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 965/1942] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:55 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:55.460760 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95490877-3a1a-4e07-9aad-7ef511da7e03 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:55.524116 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95490877-3a1a-4e07-9aad-7ef511da7e03 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:55.528349 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b91a62-65e6-4b3f-be54-d5cf5552203d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:55.529176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b91a62-65e6-4b3f-be54-d5cf5552203d service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:55.529464 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b91a62-65e6-4b3f-be54-d5cf5552203d service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:55.529737 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b91a62-65e6-4b3f-be54-d5cf5552203d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:55.540754 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b91a62-65e6-4b3f-be54-d5cf5552203d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:55.543337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b91a62-65e6-4b3f-be54-d5cf5552203d service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:55.554831 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 978/1943] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:55 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:56.410705 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e969ddad-230a-4431-b100-99d118409b87 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:56.448573 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:56.522998 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e969ddad-230a-4431-b100-99d118409b87 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:56.529649 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe3c956-b5eb-49ea-9f7d-e9bd2e19809e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:56.530805 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3c956-b5eb-49ea-9f7d-e9bd2e19809e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:56.531201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3c956-b5eb-49ea-9f7d-e9bd2e19809e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:56.531532 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3c956-b5eb-49ea-9f7d-e9bd2e19809e admin admin] PATH_INFO: `/v3/users/230f54141c364d498ee439efa2cebe16` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:56.532841 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe3c956-b5eb-49ea-9f7d-e9bd2e19809e admin admin] RBAC: Authorizing `identity:delete_user(user_id=230f54141c364d498ee439efa2cebe16)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:56.534878 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe3c956-b5eb-49ea-9f7d-e9bd2e19809e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:56.622319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8fe3c956-b5eb-49ea-9f7d-e9bd2e19809e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '230f54141c364d498ee439efa2cebe16'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:56.640935 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8fe3c956-b5eb-49ea-9f7d-e9bd2e19809e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '230f54141c364d498ee439efa2cebe16'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:56.651780 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8fe3c956-b5eb-49ea-9f7d-e9bd2e19809e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '230f54141c364d498ee439efa2cebe16'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:56.663275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8fe3c956-b5eb-49ea-9f7d-e9bd2e19809e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '230f54141c364d498ee439efa2cebe16'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:56.664758 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8fe3c956-b5eb-49ea-9f7d-e9bd2e19809e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '230f54141c364d498ee439efa2cebe16'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:56.679450 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 966/1944] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:56 2022] DELETE /identity/v3/users/230f54141c364d498ee439efa2cebe16 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:56.687149 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2900a0a-9730-4e59-8381-03ac670958ad None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:56.687149 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2900a0a-9730-4e59-8381-03ac670958ad None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:56.687903 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2900a0a-9730-4e59-8381-03ac670958ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:56.715428 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-d2900a0a-9730-4e59-8381-03ac670958ad None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:56.716731 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-d2900a0a-9730-4e59-8381-03ac670958ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:56.815251 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:56.859124 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 979/1945] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:56.877991 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe3c956-b5eb-49ea-9f7d-e9bd2e19809e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:56.878961 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8fe3c956-b5eb-49ea-9f7d-e9bd2e19809e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:57.000498 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe3c956-b5eb-49ea-9f7d-e9bd2e19809e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:57.010250 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a187b-b860-4a24-b046-0516742b302f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:57.012061 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a187b-b860-4a24-b046-0516742b302f service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:57.012823 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a187b-b860-4a24-b046-0516742b302f service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:57.013740 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a187b-b860-4a24-b046-0516742b302f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:57.024082 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2900a0a-9730-4e59-8381-03ac670958ad None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:57.031132 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-d2900a0a-9730-4e59-8381-03ac670958ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:57.046016 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a187b-b860-4a24-b046-0516742b302f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:57.049738 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a187b-b860-4a24-b046-0516742b302f service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:57.073799 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2900a0a-9730-4e59-8381-03ac670958ad None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:57.075705 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 967/1946] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:57.091939 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23619ea-f991-49b4-bee0-eb569a6b4c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:57.092908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23619ea-f991-49b4-bee0-eb569a6b4c9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:57.093606 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23619ea-f991-49b4-bee0-eb569a6b4c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:57.094029 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23619ea-f991-49b4-bee0-eb569a6b4c9c admin admin] PATH_INFO: `/v3/users/8ea55a9a2ccc4eb892865ebce29fdd68` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:57.096079 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23619ea-f991-49b4-bee0-eb569a6b4c9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ea55a9a2ccc4eb892865ebce29fdd68)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:57.098628 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23619ea-f991-49b4-bee0-eb569a6b4c9c admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:57.189514 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-d23619ea-f991-49b4-bee0-eb569a6b4c9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ea55a9a2ccc4eb892865ebce29fdd68'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:57.216897 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-d23619ea-f991-49b4-bee0-eb569a6b4c9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ea55a9a2ccc4eb892865ebce29fdd68'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:57.260606 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-d23619ea-f991-49b4-bee0-eb569a6b4c9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ea55a9a2ccc4eb892865ebce29fdd68'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:57.273867 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-d23619ea-f991-49b4-bee0-eb569a6b4c9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ea55a9a2ccc4eb892865ebce29fdd68'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:57.296306 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-d23619ea-f991-49b4-bee0-eb569a6b4c9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ea55a9a2ccc4eb892865ebce29fdd68'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:57.318892 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 980/1947] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:57 2022] DELETE /identity/v3/users/8ea55a9a2ccc4eb892865ebce29fdd68 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:57.324521 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5debc9f-424b-4bfe-9295-60ffd6fd0697 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:57.325566 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5debc9f-424b-4bfe-9295-60ffd6fd0697 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:57.326330 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5debc9f-424b-4bfe-9295-60ffd6fd0697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:57.388603 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b5debc9f-424b-4bfe-9295-60ffd6fd0697 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:57.390546 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b5debc9f-424b-4bfe-9295-60ffd6fd0697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:57.515966 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:57.569690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 968/1948] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:12:57.581543 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23619ea-f991-49b4-bee0-eb569a6b4c9c admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:57.583539 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-d23619ea-f991-49b4-bee0-eb569a6b4c9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:57.662189 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23619ea-f991-49b4-bee0-eb569a6b4c9c admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:57.667273 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683cb8d4-fcdd-4916-9913-d0f55cc45b0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:57.669364 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683cb8d4-fcdd-4916-9913-d0f55cc45b0b service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:57.669737 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683cb8d4-fcdd-4916-9913-d0f55cc45b0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:57.670165 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683cb8d4-fcdd-4916-9913-d0f55cc45b0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:57.681590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683cb8d4-fcdd-4916-9913-d0f55cc45b0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:57.683401 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683cb8d4-fcdd-4916-9913-d0f55cc45b0b service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:57.698324 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 981/1949] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:57.804784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5debc9f-424b-4bfe-9295-60ffd6fd0697 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:57.808971 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b5debc9f-424b-4bfe-9295-60ffd6fd0697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:57.826927 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5debc9f-424b-4bfe-9295-60ffd6fd0697 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:57.832867 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281acc1-c295-4d1b-b082-7412a9708da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:57.834037 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281acc1-c295-4d1b-b082-7412a9708da8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:57.834509 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281acc1-c295-4d1b-b082-7412a9708da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:57.834804 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281acc1-c295-4d1b-b082-7412a9708da8 admin admin] PATH_INFO: `/v3/projects/f2afb72b5a6f4246951d5050c4a56ff0` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:57.836152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281acc1-c295-4d1b-b082-7412a9708da8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2afb72b5a6f4246951d5050c4a56ff0)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:57.838328 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281acc1-c295-4d1b-b082-7412a9708da8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:57.923338 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4281acc1-c295-4d1b-b082-7412a9708da8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2afb72b5a6f4246951d5050c4a56ff0', 'request_id': 'req-4281acc1-c295-4d1b-b082-7412a9708da8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:57.930480 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4281acc1-c295-4d1b-b082-7412a9708da8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2afb72b5a6f4246951d5050c4a56ff0', 'request_id': 'req-4281acc1-c295-4d1b-b082-7412a9708da8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:57.940492 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4281acc1-c295-4d1b-b082-7412a9708da8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2afb72b5a6f4246951d5050c4a56ff0', 'request_id': 'req-4281acc1-c295-4d1b-b082-7412a9708da8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:57.956223 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4281acc1-c295-4d1b-b082-7412a9708da8 admin admin] The token cache is being invalidate because project f2afb72b5a6f4246951d5050c4a56ff0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:57.956589 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4281acc1-c295-4d1b-b082-7412a9708da8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:57.957543 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4281acc1-c295-4d1b-b082-7412a9708da8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:57.960602 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 969/1950] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:57 2022] DELETE /identity/v3/projects/f2afb72b5a6f4246951d5050c4a56ff0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:57.964592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683cb8d4-fcdd-4916-9913-d0f55cc45b0b service neutron] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:57.977467 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683cb8d4-fcdd-4916-9913-d0f55cc45b0b service neutron] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:58.075242 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19addb44-3a67-402a-8d74-ff5f6468405b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:58.076760 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19addb44-3a67-402a-8d74-ff5f6468405b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:58.077237 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19addb44-3a67-402a-8d74-ff5f6468405b admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:58.077693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19addb44-3a67-402a-8d74-ff5f6468405b admin admin] PATH_INFO: `/v3/users/14be012b806d43539bee74e540fe8148` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:58.079880 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19addb44-3a67-402a-8d74-ff5f6468405b admin admin] RBAC: Authorizing `identity:delete_user(user_id=14be012b806d43539bee74e540fe8148)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:58.083350 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19addb44-3a67-402a-8d74-ff5f6468405b admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:58.086318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:58.148750 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281acc1-c295-4d1b-b082-7412a9708da8 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:58.150624 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4281acc1-c295-4d1b-b082-7412a9708da8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:58.175652 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281acc1-c295-4d1b-b082-7412a9708da8 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:58.182011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c2dcf-14e8-4c2c-b684-cac0f1082615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:58.182987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c2dcf-14e8-4c2c-b684-cac0f1082615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:58.183415 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c2dcf-14e8-4c2c-b684-cac0f1082615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:58.183794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c2dcf-14e8-4c2c-b684-cac0f1082615 admin admin] PATH_INFO: `/v3/projects/9610259c00014674a535b4d08bc20e26` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:58.185355 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c2dcf-14e8-4c2c-b684-cac0f1082615 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9610259c00014674a535b4d08bc20e26)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:58.187865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c2dcf-14e8-4c2c-b684-cac0f1082615 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:58.233463 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-19addb44-3a67-402a-8d74-ff5f6468405b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14be012b806d43539bee74e540fe8148'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:58.246494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-19addb44-3a67-402a-8d74-ff5f6468405b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14be012b806d43539bee74e540fe8148'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:58.266619 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-19addb44-3a67-402a-8d74-ff5f6468405b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14be012b806d43539bee74e540fe8148'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:58.267566 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-19addb44-3a67-402a-8d74-ff5f6468405b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14be012b806d43539bee74e540fe8148'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:58.281503 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-bd0c2dcf-14e8-4c2c-b684-cac0f1082615 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9610259c00014674a535b4d08bc20e26', 'request_id': 'req-bd0c2dcf-14e8-4c2c-b684-cac0f1082615'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:58.281804 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-19addb44-3a67-402a-8d74-ff5f6468405b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14be012b806d43539bee74e540fe8148'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:58.295132 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 982/1951] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:57 2022] DELETE /identity/v3/users/14be012b806d43539bee74e540fe8148 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:12:58.311013 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-bd0c2dcf-14e8-4c2c-b684-cac0f1082615 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9610259c00014674a535b4d08bc20e26', 'request_id': 'req-bd0c2dcf-14e8-4c2c-b684-cac0f1082615'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:58.327721 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-bd0c2dcf-14e8-4c2c-b684-cac0f1082615 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9610259c00014674a535b4d08bc20e26', 'request_id': 'req-bd0c2dcf-14e8-4c2c-b684-cac0f1082615'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:58.360184 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-bd0c2dcf-14e8-4c2c-b684-cac0f1082615 admin admin] The token cache is being invalidate because project 9610259c00014674a535b4d08bc20e26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:58.360773 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-bd0c2dcf-14e8-4c2c-b684-cac0f1082615 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:58.361963 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-bd0c2dcf-14e8-4c2c-b684-cac0f1082615 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:58.366479 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 970/1952] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:58 2022] DELETE /identity/v3/projects/9610259c00014674a535b4d08bc20e26 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:58.372750 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19addb44-3a67-402a-8d74-ff5f6468405b admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:58.375425 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-19addb44-3a67-402a-8d74-ff5f6468405b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:58.494893 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19addb44-3a67-402a-8d74-ff5f6468405b admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:58.503152 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbdce16-35ae-45f9-8168-057c58146937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:58.504065 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbdce16-35ae-45f9-8168-057c58146937 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:58.504422 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbdce16-35ae-45f9-8168-057c58146937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:58.504717 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbdce16-35ae-45f9-8168-057c58146937 admin admin] PATH_INFO: `/v3/users/1e5d08f8fff84078b46e84cd795a74b9` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:58.506206 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbdce16-35ae-45f9-8168-057c58146937 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e5d08f8fff84078b46e84cd795a74b9)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:58.509045 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbdce16-35ae-45f9-8168-057c58146937 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:58.618274 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-ecbdce16-35ae-45f9-8168-057c58146937 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e5d08f8fff84078b46e84cd795a74b9'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:58.636002 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-ecbdce16-35ae-45f9-8168-057c58146937 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e5d08f8fff84078b46e84cd795a74b9'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:58.665824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-ecbdce16-35ae-45f9-8168-057c58146937 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e5d08f8fff84078b46e84cd795a74b9'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:58.667481 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-ecbdce16-35ae-45f9-8168-057c58146937 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e5d08f8fff84078b46e84cd795a74b9'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:58.681339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-ecbdce16-35ae-45f9-8168-057c58146937 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e5d08f8fff84078b46e84cd795a74b9'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:58.684630 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:58.702027 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 983/1953] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:58 2022] DELETE /identity/v3/users/1e5d08f8fff84078b46e84cd795a74b9 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:58.783574 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c2dcf-14e8-4c2c-b684-cac0f1082615 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:58.786376 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bd0c2dcf-14e8-4c2c-b684-cac0f1082615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:58.892894 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:58.931319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c2dcf-14e8-4c2c-b684-cac0f1082615 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:59.060810 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a17a5a-fc76-4f98-b99e-10275375f60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:59.062568 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a17a5a-fc76-4f98-b99e-10275375f60c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:59.064111 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a17a5a-fc76-4f98-b99e-10275375f60c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:59.064111 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a17a5a-fc76-4f98-b99e-10275375f60c admin admin] PATH_INFO: `/v3/projects/964b13e396a94991a2a48314b46b3298` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:59.067011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a17a5a-fc76-4f98-b99e-10275375f60c admin admin] RBAC: Authorizing `identity:delete_project(project_id=964b13e396a94991a2a48314b46b3298)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:59.071255 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a17a5a-fc76-4f98-b99e-10275375f60c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:59.181966 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f1a17a5a-fc76-4f98-b99e-10275375f60c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '964b13e396a94991a2a48314b46b3298', 'request_id': 'req-f1a17a5a-fc76-4f98-b99e-10275375f60c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:59.203472 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f1a17a5a-fc76-4f98-b99e-10275375f60c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '964b13e396a94991a2a48314b46b3298', 'request_id': 'req-f1a17a5a-fc76-4f98-b99e-10275375f60c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:59.221664 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f1a17a5a-fc76-4f98-b99e-10275375f60c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '964b13e396a94991a2a48314b46b3298', 'request_id': 'req-f1a17a5a-fc76-4f98-b99e-10275375f60c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:59.239385 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbdce16-35ae-45f9-8168-057c58146937 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:59.241658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-ecbdce16-35ae-45f9-8168-057c58146937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:59.258638 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f1a17a5a-fc76-4f98-b99e-10275375f60c admin admin] The token cache is being invalidate because project 964b13e396a94991a2a48314b46b3298 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:59.259160 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f1a17a5a-fc76-4f98-b99e-10275375f60c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:59.260334 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f1a17a5a-fc76-4f98-b99e-10275375f60c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:59.269082 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 971/1954] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:58 2022] DELETE /identity/v3/projects/964b13e396a94991a2a48314b46b3298 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:59.276187 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879afc5b-872b-4db2-b5bc-2e28217af526 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:59.276765 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879afc5b-872b-4db2-b5bc-2e28217af526 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:59.277752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879afc5b-872b-4db2-b5bc-2e28217af526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:59.290381 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbdce16-35ae-45f9-8168-057c58146937 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:59.311107 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-879afc5b-872b-4db2-b5bc-2e28217af526 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:59.312861 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-879afc5b-872b-4db2-b5bc-2e28217af526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:59.411415 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff9257-185d-4eac-8e8c-2e1a2b8da8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:59.413519 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff9257-185d-4eac-8e8c-2e1a2b8da8ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:59.414090 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff9257-185d-4eac-8e8c-2e1a2b8da8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:59.414843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff9257-185d-4eac-8e8c-2e1a2b8da8ff admin admin] PATH_INFO: `/v3/projects/a04432a31b874b939e284ccfedcc8a4d` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:59.417248 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff9257-185d-4eac-8e8c-2e1a2b8da8ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=a04432a31b874b939e284ccfedcc8a4d)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:59.418591 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 972/1955] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:59.428958 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff9257-185d-4eac-8e8c-2e1a2b8da8ff admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:59.430454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8520823-466c-4186-bbeb-38313e324c9a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:59.431054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8520823-466c-4186-bbeb-38313e324c9a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:59.431476 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8520823-466c-4186-bbeb-38313e324c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:59.462548 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e8520823-466c-4186-bbeb-38313e324c9a None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:59.465420 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e8520823-466c-4186-bbeb-38313e324c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:59.475122 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 973/1956] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:59.479482 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8520823-466c-4186-bbeb-38313e324c9a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:59.490327 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8520823-466c-4186-bbeb-38313e324c9a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:59.491902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-b4ff9257-185d-4eac-8e8c-2e1a2b8da8ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a04432a31b874b939e284ccfedcc8a4d', 'request_id': 'req-b4ff9257-185d-4eac-8e8c-2e1a2b8da8ff'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:59.499722 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-b4ff9257-185d-4eac-8e8c-2e1a2b8da8ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a04432a31b874b939e284ccfedcc8a4d', 'request_id': 'req-b4ff9257-185d-4eac-8e8c-2e1a2b8da8ff'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:59.514412 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:59.514756 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-b4ff9257-185d-4eac-8e8c-2e1a2b8da8ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a04432a31b874b939e284ccfedcc8a4d', 'request_id': 'req-b4ff9257-185d-4eac-8e8c-2e1a2b8da8ff'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:59.535671 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-b4ff9257-185d-4eac-8e8c-2e1a2b8da8ff admin admin] The token cache is being invalidate because project a04432a31b874b939e284ccfedcc8a4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:59.536149 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-b4ff9257-185d-4eac-8e8c-2e1a2b8da8ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:59.537178 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-b4ff9257-185d-4eac-8e8c-2e1a2b8da8ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:59.539941 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 984/1957] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:59 2022] DELETE /identity/v3/projects/a04432a31b874b939e284ccfedcc8a4d => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:59.549522 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16864dc1-4e91-489a-bc8f-f1bccb97c415 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:59.550235 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16864dc1-4e91-489a-bc8f-f1bccb97c415 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:59.550758 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16864dc1-4e91-489a-bc8f-f1bccb97c415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:59.585869 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-16864dc1-4e91-489a-bc8f-f1bccb97c415 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:59.588347 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-16864dc1-4e91-489a-bc8f-f1bccb97c415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:59.594549 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842d489-79ef-4563-88b2-a94d7b0c958a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:59.595686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842d489-79ef-4563-88b2-a94d7b0c958a admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:59.596180 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842d489-79ef-4563-88b2-a94d7b0c958a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:59.596675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842d489-79ef-4563-88b2-a94d7b0c958a admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:59.598085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1842d489-79ef-4563-88b2-a94d7b0c958a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:12:59.598643 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1842d489-79ef-4563-88b2-a94d7b0c958a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:59.601497 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1842d489-79ef-4563-88b2-a94d7b0c958a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:59.618280 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 985/1958] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:12:59.621321 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25322478-78be-4ef6-9191-5a12dd55ed98 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:59.621756 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25322478-78be-4ef6-9191-5a12dd55ed98 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:59.622301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25322478-78be-4ef6-9191-5a12dd55ed98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:59.637105 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 974/1959] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:12:59.639751 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842d489-79ef-4563-88b2-a94d7b0c958a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:59.643011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1842d489-79ef-4563-88b2-a94d7b0c958a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:59.650971 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-25322478-78be-4ef6-9191-5a12dd55ed98 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:59.652331 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-25322478-78be-4ef6-9191-5a12dd55ed98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:59.660804 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 986/1960] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:12:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:12:59.665341 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25322478-78be-4ef6-9191-5a12dd55ed98 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:59.668825 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842d489-79ef-4563-88b2-a94d7b0c958a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:59.680796 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25322478-78be-4ef6-9191-5a12dd55ed98 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:59.691890 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9acc3c8-212e-4c2f-86a4-49010b7b3b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:59.693260 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acc3c8-212e-4c2f-86a4-49010b7b3b14 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:59.693798 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acc3c8-212e-4c2f-86a4-49010b7b3b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:59.694421 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acc3c8-212e-4c2f-86a4-49010b7b3b14 admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:59.695840 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb5c82-755c-4eb6-84eb-3a503810e6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:59.696012 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9acc3c8-212e-4c2f-86a4-49010b7b3b14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:12:59.696434 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9acc3c8-212e-4c2f-86a4-49010b7b3b14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:59.696968 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb5c82-755c-4eb6-84eb-3a503810e6b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:59.697464 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb5c82-755c-4eb6-84eb-3a503810e6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:59.697967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb5c82-755c-4eb6-84eb-3a503810e6b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:59.700390 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb5c82-755c-4eb6-84eb-3a503810e6b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:59.700596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9acc3c8-212e-4c2f-86a4-49010b7b3b14 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:59.702510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb5c82-755c-4eb6-84eb-3a503810e6b5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:59.727889 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 987/1961] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:12:59.731490 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9acc3c8-212e-4c2f-86a4-49010b7b3b14 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:59.750434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 975/1962] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:59 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:59.751303 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9acc3c8-212e-4c2f-86a4-49010b7b3b14 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:59.751712 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb5c82-755c-4eb6-84eb-3a503810e6b5 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:59.830619 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e91e07-700f-44eb-acd6-05d47ca6eb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:59.831496 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e91e07-700f-44eb-acd6-05d47ca6eb37 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:59.832148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e91e07-700f-44eb-acd6-05d47ca6eb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:59.832410 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e91e07-700f-44eb-acd6-05d47ca6eb37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:59.833418 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e91e07-700f-44eb-acd6-05d47ca6eb37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:59.835218 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e91e07-700f-44eb-acd6-05d47ca6eb37 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:59.841261 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:59.843607 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb5c82-755c-4eb6-84eb-3a503810e6b5 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:59.852778 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdffe31-91e3-4f24-9d82-5b050a1419f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:59.854021 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdffe31-91e3-4f24-9d82-5b050a1419f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:59.854815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdffe31-91e3-4f24-9d82-5b050a1419f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:59.855525 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdffe31-91e3-4f24-9d82-5b050a1419f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:59.856727 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdffe31-91e3-4f24-9d82-5b050a1419f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:59.861034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdffe31-91e3-4f24-9d82-5b050a1419f9 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:59.877559 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 988/1963] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:59 2022] POST /identity/v3/projects => generated 381 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:59.888702 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e91e07-700f-44eb-acd6-05d47ca6eb37 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:59.901368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 976/1964] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:59 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:59.908098 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdffe31-91e3-4f24-9d82-5b050a1419f9 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:59.928687 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdffe31-91e3-4f24-9d82-5b050a1419f9 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:00.010824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e91e07-700f-44eb-acd6-05d47ca6eb37 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:00.018372 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28fd444-e999-43bb-acf6-76ea1ab71305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:00.019221 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fd444-e999-43bb-acf6-76ea1ab71305 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:00.019606 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fd444-e999-43bb-acf6-76ea1ab71305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:00.021527 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fd444-e999-43bb-acf6-76ea1ab71305 admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:00.021527 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28fd444-e999-43bb-acf6-76ea1ab71305 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:00.023347 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28fd444-e999-43bb-acf6-76ea1ab71305 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:00.031876 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ca4b0-7469-44a5-ace8-2d2c9fb98c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:00.033014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ca4b0-7469-44a5-ace8-2d2c9fb98c42 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:00.033474 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ca4b0-7469-44a5-ace8-2d2c9fb98c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:00.033897 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ca4b0-7469-44a5-ace8-2d2c9fb98c42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:00.035770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ca4b0-7469-44a5-ace8-2d2c9fb98c42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:00.038843 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ca4b0-7469-44a5-ace8-2d2c9fb98c42 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:00.055809 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 977/1965] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:13:00.056084 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 989/1966] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:59 2022] POST /identity/v3/users => generated 291 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:00.061545 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28fd444-e999-43bb-acf6-76ea1ab71305 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:00.064763 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ca4b0-7469-44a5-ace8-2d2c9fb98c42 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:00.076083 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28fd444-e999-43bb-acf6-76ea1ab71305 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:00.082735 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a29e2f9-7b30-46cb-ba8e-d7943ab21094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:00.083677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ca4b0-7469-44a5-ace8-2d2c9fb98c42 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:00.083892 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29e2f9-7b30-46cb-ba8e-d7943ab21094 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:00.084251 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29e2f9-7b30-46cb-ba8e-d7943ab21094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:00.084732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29e2f9-7b30-46cb-ba8e-d7943ab21094 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:00.086132 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a29e2f9-7b30-46cb-ba8e-d7943ab21094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:00.090429 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a29e2f9-7b30-46cb-ba8e-d7943ab21094 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:00.093156 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c92d86-ce67-4d61-b4de-c4b91307bfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:00.094619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c92d86-ce67-4d61-b4de-c4b91307bfe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:00.095238 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c92d86-ce67-4d61-b4de-c4b91307bfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:00.095812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c92d86-ce67-4d61-b4de-c4b91307bfe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:00.097285 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c92d86-ce67-4d61-b4de-c4b91307bfe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:00.100616 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c92d86-ce67-4d61-b4de-c4b91307bfe9 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:00.114113 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 990/1967] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:00.120413 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a29e2f9-7b30-46cb-ba8e-d7943ab21094 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:00.129023 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 978/1968] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:00.134961 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c92d86-ce67-4d61-b4de-c4b91307bfe9 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:00.145440 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:00.152597 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a29e2f9-7b30-46cb-ba8e-d7943ab21094 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:00.155115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c92d86-ce67-4d61-b4de-c4b91307bfe9 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:00.161665 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e1b0a-cbb3-411e-a164-c1b9b8bb7a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:00.163339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e1b0a-cbb3-411e-a164-c1b9b8bb7a00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:00.163843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e1b0a-cbb3-411e-a164-c1b9b8bb7a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:00.164341 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e1b0a-cbb3-411e-a164-c1b9b8bb7a00 admin admin] PATH_INFO: `/v3/projects/2c4305bcb3af4a92b109f25f8f0bb382/users/7ce0d7fca12a4e2aa11f0c193624c6f7/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:00.165824 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39210291-3b11-493a-ab85-18d220418bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:00.167007 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39210291-3b11-493a-ab85-18d220418bb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:00.167867 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39210291-3b11-493a-ab85-18d220418bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:00.168376 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39210291-3b11-493a-ab85-18d220418bb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:00.169531 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39210291-3b11-493a-ab85-18d220418bb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:00.171729 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39210291-3b11-493a-ab85-18d220418bb7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:00.187353 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 979/1969] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:13:00.201353 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39210291-3b11-493a-ab85-18d220418bb7 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:00.208151 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e1b0a-cbb3-411e-a164-c1b9b8bb7a00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c4305bcb3af4a92b109f25f8f0bb382, user_id=7ce0d7fca12a4e2aa11f0c193624c6f7, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:00.211418 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e1b0a-cbb3-411e-a164-c1b9b8bb7a00 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:00.225310 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39210291-3b11-493a-ab85-18d220418bb7 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:00.235736 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 991/1970] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:00 2022] PUT /identity/v3/projects/2c4305bcb3af4a92b109f25f8f0bb382/users/7ce0d7fca12a4e2aa11f0c193624c6f7/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:00.243679 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e5003-698d-4326-8ab3-9df0c17e56ad None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:00.245025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e5003-698d-4326-8ab3-9df0c17e56ad None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:00.245025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e5003-698d-4326-8ab3-9df0c17e56ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:00.275865 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-022e5003-698d-4326-8ab3-9df0c17e56ad None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:00.278295 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-022e5003-698d-4326-8ab3-9df0c17e56ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:00.354552 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe3d6b-807f-4401-bd4b-01979e60efc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:00.356732 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe3d6b-807f-4401-bd4b-01979e60efc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:00.357050 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe3d6b-807f-4401-bd4b-01979e60efc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:00.357332 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe3d6b-807f-4401-bd4b-01979e60efc9 admin admin] PATH_INFO: `/v3/projects/49f6ccb7749e4b3db8f264c1e137f7f8/users/af16b7af7d4b4e758a0f9ff26d03376e/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:00.380859 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebe3d6b-807f-4401-bd4b-01979e60efc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49f6ccb7749e4b3db8f264c1e137f7f8, user_id=af16b7af7d4b4e758a0f9ff26d03376e, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:00.384005 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebe3d6b-807f-4401-bd4b-01979e60efc9 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:00.407897 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 980/1971] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:00 2022] PUT /identity/v3/projects/49f6ccb7749e4b3db8f264c1e137f7f8/users/af16b7af7d4b4e758a0f9ff26d03376e/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:00.409617 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 992/1972] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:00.434443 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89268a8-dfb4-4b34-b67e-e39b8f542eb8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:00.435513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89268a8-dfb4-4b34-b67e-e39b8f542eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:00.436456 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e5003-698d-4326-8ab3-9df0c17e56ad None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:00.436850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89268a8-dfb4-4b34-b67e-e39b8f542eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:00.439283 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-022e5003-698d-4326-8ab3-9df0c17e56ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:00.475471 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d89268a8-dfb4-4b34-b67e-e39b8f542eb8 None None] MFA Rules not processed for user `af16b7af7d4b4e758a0f9ff26d03376e`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:00.477531 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d89268a8-dfb4-4b34-b67e-e39b8f542eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:00.532338 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:00.561984 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e5003-698d-4326-8ab3-9df0c17e56ad None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:00.600727 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 981/1973] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:13:00.602852 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89268a8-dfb4-4b34-b67e-e39b8f542eb8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:00.675591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db9449-0003-459c-8ce3-4d1c81007adb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:00.677105 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db9449-0003-459c-8ce3-4d1c81007adb service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:00.677697 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db9449-0003-459c-8ce3-4d1c81007adb service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:00.680054 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db9449-0003-459c-8ce3-4d1c81007adb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:00.722347 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89268a8-dfb4-4b34-b67e-e39b8f542eb8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:00.730967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca77fb3-bbd1-4c77-ae32-63cc34772642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:00.731822 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca77fb3-bbd1-4c77-ae32-63cc34772642 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:00.732168 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca77fb3-bbd1-4c77-ae32-63cc34772642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:00.732512 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca77fb3-bbd1-4c77-ae32-63cc34772642 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:00.733297 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca77fb3-bbd1-4c77-ae32-63cc34772642 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:00.735272 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca77fb3-bbd1-4c77-ae32-63cc34772642 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:00.748990 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:00.772780 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db9449-0003-459c-8ce3-4d1c81007adb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:00.774327 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db9449-0003-459c-8ce3-4d1c81007adb service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:00.775605 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 982/1974] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:00 2022] POST /identity/v3/projects => generated 379 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:00.780922 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca77fb3-bbd1-4c77-ae32-63cc34772642 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:00.787622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 993/1975] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:00.883035 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca77fb3-bbd1-4c77-ae32-63cc34772642 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:00.893744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3297c91-9776-4c72-a20d-4349e8311bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:00.895070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3297c91-9776-4c72-a20d-4349e8311bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:00.895592 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3297c91-9776-4c72-a20d-4349e8311bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:00.896079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3297c91-9776-4c72-a20d-4349e8311bd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:00.897181 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3297c91-9776-4c72-a20d-4349e8311bd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:00.899591 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3297c91-9776-4c72-a20d-4349e8311bd0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:00.927384 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 983/1976] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:00 2022] POST /identity/v3/users => generated 296 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:13:00.934459 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db9449-0003-459c-8ce3-4d1c81007adb service neutron] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:00.950927 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db9449-0003-459c-8ce3-4d1c81007adb service neutron] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:00.958938 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d0b54-5310-4733-a587-826b1b15261d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:00.959760 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d0b54-5310-4733-a587-826b1b15261d admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:00.960158 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d0b54-5310-4733-a587-826b1b15261d admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:00.960528 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d0b54-5310-4733-a587-826b1b15261d admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:00.961529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82d0b54-5310-4733-a587-826b1b15261d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:00.963938 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82d0b54-5310-4733-a587-826b1b15261d admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:00.981026 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 994/1977] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:00.986954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3297c91-9776-4c72-a20d-4349e8311bd0 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:01.002753 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3297c91-9776-4c72-a20d-4349e8311bd0 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:01.010783 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25597c6a-0ac9-4ce5-9383-cf8b091128f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:01.012244 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25597c6a-0ac9-4ce5-9383-cf8b091128f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:01.012659 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25597c6a-0ac9-4ce5-9383-cf8b091128f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:01.012859 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25597c6a-0ac9-4ce5-9383-cf8b091128f3 admin admin] PATH_INFO: `/v3/projects/ee22cee442c244d9b0372a5a0b7f465b/users/f6111fa271634c5a987f617627579e9e/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:01.040036 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25597c6a-0ac9-4ce5-9383-cf8b091128f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee22cee442c244d9b0372a5a0b7f465b, user_id=f6111fa271634c5a987f617627579e9e, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:01.046316 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25597c6a-0ac9-4ce5-9383-cf8b091128f3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:01.076126 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 984/1978] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:00 2022] PUT /identity/v3/projects/ee22cee442c244d9b0372a5a0b7f465b/users/f6111fa271634c5a987f617627579e9e/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:01.081605 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d0b54-5310-4733-a587-826b1b15261d admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:01.094714 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d0b54-5310-4733-a587-826b1b15261d admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:01.204769 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ddd4c-5c76-4a2f-bc72-0dec51de9f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:01.205886 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ddd4c-5c76-4a2f-bc72-0dec51de9f9c admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:01.206771 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ddd4c-5c76-4a2f-bc72-0dec51de9f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:01.207270 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ddd4c-5c76-4a2f-bc72-0dec51de9f9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:01.208805 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ddd4c-5c76-4a2f-bc72-0dec51de9f9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:01.212505 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ddd4c-5c76-4a2f-bc72-0dec51de9f9c admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:01.234085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 995/1979] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:01.240261 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25597c6a-0ac9-4ce5-9383-cf8b091128f3 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:01.258435 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25597c6a-0ac9-4ce5-9383-cf8b091128f3 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:01.267226 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212755a-4ff1-48a3-8155-908d328109d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:01.268969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212755a-4ff1-48a3-8155-908d328109d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:01.269386 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212755a-4ff1-48a3-8155-908d328109d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:01.270259 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212755a-4ff1-48a3-8155-908d328109d1 admin admin] PATH_INFO: `/v3/projects/ee22cee442c244d9b0372a5a0b7f465b/users/f6111fa271634c5a987f617627579e9e/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:01.273390 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5212755a-4ff1-48a3-8155-908d328109d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee22cee442c244d9b0372a5a0b7f465b, user_id=f6111fa271634c5a987f617627579e9e, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:01.279399 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5212755a-4ff1-48a3-8155-908d328109d1 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:01.301675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 985/1980] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:01 2022] PUT /identity/v3/projects/ee22cee442c244d9b0372a5a0b7f465b/users/f6111fa271634c5a987f617627579e9e/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:01.319837 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653867a-dbf1-4d2d-a96d-b402788ec238 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:01.320296 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653867a-dbf1-4d2d-a96d-b402788ec238 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:01.320796 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653867a-dbf1-4d2d-a96d-b402788ec238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:01.408805 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-8653867a-dbf1-4d2d-a96d-b402788ec238 None None] MFA Rules not processed for user `f6111fa271634c5a987f617627579e9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:01.410483 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-8653867a-dbf1-4d2d-a96d-b402788ec238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:01.474310 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:01.622583 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 996/1981] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:01 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:01.643163 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212755a-4ff1-48a3-8155-908d328109d1 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:01.646390 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5212755a-4ff1-48a3-8155-908d328109d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:01.841578 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212755a-4ff1-48a3-8155-908d328109d1 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:01.849396 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed46eb0-135a-435e-8267-5d0b4d4e1eb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:01.850703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed46eb0-135a-435e-8267-5d0b4d4e1eb6 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:01.851294 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed46eb0-135a-435e-8267-5d0b4d4e1eb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:01.851928 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed46eb0-135a-435e-8267-5d0b4d4e1eb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:01.875438 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed46eb0-135a-435e-8267-5d0b4d4e1eb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:01.882043 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed46eb0-135a-435e-8267-5d0b4d4e1eb6 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:01.913147 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 986/1982] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:01 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:02.467559 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d6806-d394-4d96-b515-cb7924ab1758 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:02.468328 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d6806-d394-4d96-b515-cb7924ab1758 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:02.468574 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d6806-d394-4d96-b515-cb7924ab1758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:02.567692 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-bd5d6806-d394-4d96-b515-cb7924ab1758 None None] MFA Rules not processed for user `7ce0d7fca12a4e2aa11f0c193624c6f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:02.570808 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-bd5d6806-d394-4d96-b515-cb7924ab1758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:02.797794 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 997/1983] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:02.810703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed46eb0-135a-435e-8267-5d0b4d4e1eb6 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:02.937299 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:02.991670 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed46eb0-135a-435e-8267-5d0b4d4e1eb6 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:03.016604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819967bd-a3eb-4c56-a4ae-2ff763f76996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:03.018035 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819967bd-a3eb-4c56-a4ae-2ff763f76996 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:03.018925 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819967bd-a3eb-4c56-a4ae-2ff763f76996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:03.029168 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819967bd-a3eb-4c56-a4ae-2ff763f76996 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:03.029624 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819967bd-a3eb-4c56-a4ae-2ff763f76996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:03.035234 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819967bd-a3eb-4c56-a4ae-2ff763f76996 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:03.119513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 987/1984] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:02 2022] POST /identity/v3/projects => generated 373 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:03.126295 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d6806-d394-4d96-b515-cb7924ab1758 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:03.147883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d6806-d394-4d96-b515-cb7924ab1758 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:03.327535 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf5bb9-e6dd-44e6-97dc-fed6d6d4929a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:03.328670 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf5bb9-e6dd-44e6-97dc-fed6d6d4929a admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:03.328976 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf5bb9-e6dd-44e6-97dc-fed6d6d4929a admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:03.329288 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf5bb9-e6dd-44e6-97dc-fed6d6d4929a admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:03.330121 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf5bb9-e6dd-44e6-97dc-fed6d6d4929a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:03.336635 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf5bb9-e6dd-44e6-97dc-fed6d6d4929a admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:03.399341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819967bd-a3eb-4c56-a4ae-2ff763f76996 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:03.414664 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 998/1985] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:03 2022] POST /identity/v3/users => generated 293 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:13:03.416352 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf5bb9-e6dd-44e6-97dc-fed6d6d4929a admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:03.435111 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819967bd-a3eb-4c56-a4ae-2ff763f76996 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:03.437173 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf5bb9-e6dd-44e6-97dc-fed6d6d4929a admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:03.447776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1651fc-3832-49fe-b9fa-5baad4c02af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:03.449599 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1651fc-3832-49fe-b9fa-5baad4c02af7 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:03.450629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1651fc-3832-49fe-b9fa-5baad4c02af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:03.450993 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1651fc-3832-49fe-b9fa-5baad4c02af7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:03.452180 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1651fc-3832-49fe-b9fa-5baad4c02af7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:03.454194 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1651fc-3832-49fe-b9fa-5baad4c02af7 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:03.460235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa902f54-1428-499e-a580-f14002461bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:03.462702 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa902f54-1428-499e-a580-f14002461bc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:03.463241 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa902f54-1428-499e-a580-f14002461bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:03.463599 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa902f54-1428-499e-a580-f14002461bc3 admin admin] PATH_INFO: `/v3/users/af16b7af7d4b4e758a0f9ff26d03376e` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:03.473311 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa902f54-1428-499e-a580-f14002461bc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af16b7af7d4b4e758a0f9ff26d03376e)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:03.480859 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa902f54-1428-499e-a580-f14002461bc3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:03.496177 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 999/1986] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:03.499648 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1651fc-3832-49fe-b9fa-5baad4c02af7 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:03.521679 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1651fc-3832-49fe-b9fa-5baad4c02af7 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:03.535049 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec12af-0074-4cce-942c-1d5d757091d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:03.538111 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec12af-0074-4cce-942c-1d5d757091d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:03.538714 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec12af-0074-4cce-942c-1d5d757091d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:03.539738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec12af-0074-4cce-942c-1d5d757091d7 admin admin] PATH_INFO: `/v3/projects/09e52c177a144479a203c60a4ce16cdd/users/6de40790b903490dbcc29e10e48d0ac4/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:03.585687 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec12af-0074-4cce-942c-1d5d757091d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09e52c177a144479a203c60a4ce16cdd, user_id=6de40790b903490dbcc29e10e48d0ac4, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:03.594694 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec12af-0074-4cce-942c-1d5d757091d7 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:03.628758 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1000/1987] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:03 2022] PUT /identity/v3/projects/09e52c177a144479a203c60a4ce16cdd/users/6de40790b903490dbcc29e10e48d0ac4/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:03.630383 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec12af-0074-4cce-942c-1d5d757091d7 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:03.636188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fa902f54-1428-499e-a580-f14002461bc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af16b7af7d4b4e758a0f9ff26d03376e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:03.652835 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:03.678439 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fa902f54-1428-499e-a580-f14002461bc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af16b7af7d4b4e758a0f9ff26d03376e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:03.696064 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fa902f54-1428-499e-a580-f14002461bc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af16b7af7d4b4e758a0f9ff26d03376e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:03.717081 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fa902f54-1428-499e-a580-f14002461bc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af16b7af7d4b4e758a0f9ff26d03376e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:03.718953 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fa902f54-1428-499e-a580-f14002461bc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af16b7af7d4b4e758a0f9ff26d03376e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:03.731958 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 988/1988] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:03 2022] DELETE /identity/v3/users/af16b7af7d4b4e758a0f9ff26d03376e => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:03.739116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51113b4c-9b4f-43a7-ac7b-df44fe3c956c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:03.740152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51113b4c-9b4f-43a7-ac7b-df44fe3c956c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:03.741031 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51113b4c-9b4f-43a7-ac7b-df44fe3c956c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:03.773735 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-51113b4c-9b4f-43a7-ac7b-df44fe3c956c None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:03.778940 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-51113b4c-9b4f-43a7-ac7b-df44fe3c956c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:03.869815 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec12af-0074-4cce-942c-1d5d757091d7 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:03.965837 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 989/1989] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:03.976415 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51113b4c-9b4f-43a7-ac7b-df44fe3c956c None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:03.978341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-51113b4c-9b4f-43a7-ac7b-df44fe3c956c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:04.029117 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8effb46f-d7ab-4d9f-ab69-256a81f7a2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:04.030622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effb46f-d7ab-4d9f-ab69-256a81f7a2de admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:04.031113 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effb46f-d7ab-4d9f-ab69-256a81f7a2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:04.033189 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effb46f-d7ab-4d9f-ab69-256a81f7a2de admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:04.036268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8effb46f-d7ab-4d9f-ab69-256a81f7a2de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:04.037970 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8effb46f-d7ab-4d9f-ab69-256a81f7a2de admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:04.069587 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1001/1990] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:04.077070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8effb46f-d7ab-4d9f-ab69-256a81f7a2de admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:04.082252 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-8effb46f-d7ab-4d9f-ab69-256a81f7a2de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:04.095220 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:04.129715 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51113b4c-9b4f-43a7-ac7b-df44fe3c956c None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:04.131193 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8effb46f-d7ab-4d9f-ab69-256a81f7a2de admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:04.140141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d25cc8-ac39-41b6-9edf-798976729ee4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:04.140792 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d25cc8-ac39-41b6-9edf-798976729ee4 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:04.141825 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab27fd-d006-445c-8fd8-de36ea8104e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:04.142771 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab27fd-d006-445c-8fd8-de36ea8104e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:04.143102 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab27fd-d006-445c-8fd8-de36ea8104e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:04.143367 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab27fd-d006-445c-8fd8-de36ea8104e3 admin admin] PATH_INFO: `/v3/projects/09e52c177a144479a203c60a4ce16cdd/users/6de40790b903490dbcc29e10e48d0ac4/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:04.144383 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d25cc8-ac39-41b6-9edf-798976729ee4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:04.145210 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d25cc8-ac39-41b6-9edf-798976729ee4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:04.157633 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cab27fd-d006-445c-8fd8-de36ea8104e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09e52c177a144479a203c60a4ce16cdd, user_id=6de40790b903490dbcc29e10e48d0ac4, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:04.162704 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d25cc8-ac39-41b6-9edf-798976729ee4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:04.164545 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d25cc8-ac39-41b6-9edf-798976729ee4 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:04.178894 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cab27fd-d006-445c-8fd8-de36ea8104e3 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:04.188734 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 990/1991] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:13:04.216056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1002/1992] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:04 2022] PUT /identity/v3/projects/09e52c177a144479a203c60a4ce16cdd/users/6de40790b903490dbcc29e10e48d0ac4/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:04.991937 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d25cc8-ac39-41b6-9edf-798976729ee4 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:04.994817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-24d25cc8-ac39-41b6-9edf-798976729ee4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:05.019581 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d25cc8-ac39-41b6-9edf-798976729ee4 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:05.135081 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3fa42-81da-4ed7-9c79-3081f4406412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:05.136762 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3fa42-81da-4ed7-9c79-3081f4406412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:05.137365 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3fa42-81da-4ed7-9c79-3081f4406412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:05.138116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3fa42-81da-4ed7-9c79-3081f4406412 admin admin] PATH_INFO: `/v3/projects/49f6ccb7749e4b3db8f264c1e137f7f8` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:05.141288 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe3fa42-81da-4ed7-9c79-3081f4406412 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49f6ccb7749e4b3db8f264c1e137f7f8)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:05.144734 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe3fa42-81da-4ed7-9c79-3081f4406412 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:05.247149 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0fe3fa42-81da-4ed7-9c79-3081f4406412 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49f6ccb7749e4b3db8f264c1e137f7f8', 'request_id': 'req-0fe3fa42-81da-4ed7-9c79-3081f4406412'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:05.255201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0fe3fa42-81da-4ed7-9c79-3081f4406412 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49f6ccb7749e4b3db8f264c1e137f7f8', 'request_id': 'req-0fe3fa42-81da-4ed7-9c79-3081f4406412'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:05.270328 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0fe3fa42-81da-4ed7-9c79-3081f4406412 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49f6ccb7749e4b3db8f264c1e137f7f8', 'request_id': 'req-0fe3fa42-81da-4ed7-9c79-3081f4406412'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:05.297095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0fe3fa42-81da-4ed7-9c79-3081f4406412 admin admin] The token cache is being invalidate because project 49f6ccb7749e4b3db8f264c1e137f7f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:05.297678 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0fe3fa42-81da-4ed7-9c79-3081f4406412 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:05.299483 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0fe3fa42-81da-4ed7-9c79-3081f4406412 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:05.306761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 991/1993] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:04 2022] DELETE /identity/v3/projects/49f6ccb7749e4b3db8f264c1e137f7f8 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:05.307760 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab27fd-d006-445c-8fd8-de36ea8104e3 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:05.311350 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-4cab27fd-d006-445c-8fd8-de36ea8104e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:05.347716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab27fd-d006-445c-8fd8-de36ea8104e3 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:05.445730 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506aa5d-7a8a-4d5c-a507-f98c3a96c969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:05.447179 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506aa5d-7a8a-4d5c-a507-f98c3a96c969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:05.447680 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506aa5d-7a8a-4d5c-a507-f98c3a96c969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:05.448196 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506aa5d-7a8a-4d5c-a507-f98c3a96c969 admin admin] PATH_INFO: `/v3/users/f6111fa271634c5a987f617627579e9e` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:05.451230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506aa5d-7a8a-4d5c-a507-f98c3a96c969 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6111fa271634c5a987f617627579e9e)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:05.454702 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506aa5d-7a8a-4d5c-a507-f98c3a96c969 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:05.529692 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:05.549113 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7506aa5d-7a8a-4d5c-a507-f98c3a96c969 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6111fa271634c5a987f617627579e9e'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:05.561823 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7506aa5d-7a8a-4d5c-a507-f98c3a96c969 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6111fa271634c5a987f617627579e9e'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:05.582746 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7506aa5d-7a8a-4d5c-a507-f98c3a96c969 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6111fa271634c5a987f617627579e9e'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:05.584129 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7506aa5d-7a8a-4d5c-a507-f98c3a96c969 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6111fa271634c5a987f617627579e9e'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:05.605086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7506aa5d-7a8a-4d5c-a507-f98c3a96c969 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6111fa271634c5a987f617627579e9e'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:05.615806 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1003/1994] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:05 2022] DELETE /identity/v3/users/f6111fa271634c5a987f617627579e9e => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:05.824367 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527f3de-c59d-43c0-8c99-3918efa70fb3 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:05.824868 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527f3de-c59d-43c0-8c99-3918efa70fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:05.825379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527f3de-c59d-43c0-8c99-3918efa70fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:05.888880 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506aa5d-7a8a-4d5c-a507-f98c3a96c969 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:05.891145 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-7506aa5d-7a8a-4d5c-a507-f98c3a96c969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:05.899221 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-3527f3de-c59d-43c0-8c99-3918efa70fb3 None None] MFA Rules not processed for user `6de40790b903490dbcc29e10e48d0ac4`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:05.901050 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3527f3de-c59d-43c0-8c99-3918efa70fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:06.041301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506aa5d-7a8a-4d5c-a507-f98c3a96c969 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:06.050087 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbfff94-5101-40d2-9e9b-4ad092bb063d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:06.052392 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfff94-5101-40d2-9e9b-4ad092bb063d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:06.052936 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfff94-5101-40d2-9e9b-4ad092bb063d admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:06.053376 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfff94-5101-40d2-9e9b-4ad092bb063d admin admin] PATH_INFO: `/v3/users/e792b600ea114bb0bc76bda235016377` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:06.055842 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbfff94-5101-40d2-9e9b-4ad092bb063d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e792b600ea114bb0bc76bda235016377)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:06.059861 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbfff94-5101-40d2-9e9b-4ad092bb063d admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:06.137203 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:06.160288 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 992/1995] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:05 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:06.160616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7fbfff94-5101-40d2-9e9b-4ad092bb063d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e792b600ea114bb0bc76bda235016377'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:06.170515 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7fbfff94-5101-40d2-9e9b-4ad092bb063d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e792b600ea114bb0bc76bda235016377'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:06.180247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527f3de-c59d-43c0-8c99-3918efa70fb3 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:06.183405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3527f3de-c59d-43c0-8c99-3918efa70fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:06.201464 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7fbfff94-5101-40d2-9e9b-4ad092bb063d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e792b600ea114bb0bc76bda235016377'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:06.203065 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7fbfff94-5101-40d2-9e9b-4ad092bb063d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e792b600ea114bb0bc76bda235016377'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:06.223859 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7fbfff94-5101-40d2-9e9b-4ad092bb063d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e792b600ea114bb0bc76bda235016377'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:06.243642 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1004/1996] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:05 2022] DELETE /identity/v3/users/e792b600ea114bb0bc76bda235016377 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:06.251961 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17343c-fd43-467c-b254-ad6ca050470e None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:06.252553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17343c-fd43-467c-b254-ad6ca050470e None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:06.253454 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17343c-fd43-467c-b254-ad6ca050470e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:06.305779 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-fb17343c-fd43-467c-b254-ad6ca050470e None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:06.310563 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-fb17343c-fd43-467c-b254-ad6ca050470e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:06.343599 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527f3de-c59d-43c0-8c99-3918efa70fb3 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:06.429070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:06.531264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1005/1997] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:06.535362 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3781a82-72c9-44ca-b2c6-268d4d7898cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:06.536290 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17343c-fd43-467c-b254-ad6ca050470e None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:06.537527 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3781a82-72c9-44ca-b2c6-268d4d7898cc service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:06.538557 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-fb17343c-fd43-467c-b254-ad6ca050470e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:06.539068 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3781a82-72c9-44ca-b2c6-268d4d7898cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:06.539978 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3781a82-72c9-44ca-b2c6-268d4d7898cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:06.549730 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e3781a82-72c9-44ca-b2c6-268d4d7898cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:06.559932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17343c-fd43-467c-b254-ad6ca050470e None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:06.587253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb109e3-a579-48f8-881d-345086c6fb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:06.589031 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb109e3-a579-48f8-881d-345086c6fb40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:06.593524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb109e3-a579-48f8-881d-345086c6fb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:06.594743 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb109e3-a579-48f8-881d-345086c6fb40 admin admin] PATH_INFO: `/v3/projects/ee22cee442c244d9b0372a5a0b7f465b` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:06.597485 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb109e3-a579-48f8-881d-345086c6fb40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee22cee442c244d9b0372a5a0b7f465b)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:06.603406 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb109e3-a579-48f8-881d-345086c6fb40 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:06.662628 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3781a82-72c9-44ca-b2c6-268d4d7898cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:06.662628 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3781a82-72c9-44ca-b2c6-268d4d7898cc service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:06.676501 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 993/1998] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:06.677487 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3781a82-72c9-44ca-b2c6-268d4d7898cc service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:06.678981 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e3781a82-72c9-44ca-b2c6-268d4d7898cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:06.757989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-ffb109e3-a579-48f8-881d-345086c6fb40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee22cee442c244d9b0372a5a0b7f465b', 'request_id': 'req-ffb109e3-a579-48f8-881d-345086c6fb40'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:06.795558 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-ffb109e3-a579-48f8-881d-345086c6fb40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee22cee442c244d9b0372a5a0b7f465b', 'request_id': 'req-ffb109e3-a579-48f8-881d-345086c6fb40'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:06.809497 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3781a82-72c9-44ca-b2c6-268d4d7898cc service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:06.809692 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-ffb109e3-a579-48f8-881d-345086c6fb40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee22cee442c244d9b0372a5a0b7f465b', 'request_id': 'req-ffb109e3-a579-48f8-881d-345086c6fb40'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:06.845458 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-ffb109e3-a579-48f8-881d-345086c6fb40 admin admin] The token cache is being invalidate because project ee22cee442c244d9b0372a5a0b7f465b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:06.846073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-ffb109e3-a579-48f8-881d-345086c6fb40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:06.849147 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-ffb109e3-a579-48f8-881d-345086c6fb40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:06.852657 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1006/1999] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:06 2022] DELETE /identity/v3/projects/ee22cee442c244d9b0372a5a0b7f465b => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:06.861486 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b626a7f-7df8-49eb-9484-0fc5b0d48ab5 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:06.861877 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b626a7f-7df8-49eb-9484-0fc5b0d48ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:06.862250 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b626a7f-7df8-49eb-9484-0fc5b0d48ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:06.876638 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:06.888150 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-2b626a7f-7df8-49eb-9484-0fc5b0d48ab5 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:06.889396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-2b626a7f-7df8-49eb-9484-0fc5b0d48ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:06.937220 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74874545-5d76-4b3a-9a71-cc2fb7fd9c90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:06.938499 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74874545-5d76-4b3a-9a71-cc2fb7fd9c90 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:06.939088 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74874545-5d76-4b3a-9a71-cc2fb7fd9c90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:06.939603 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74874545-5d76-4b3a-9a71-cc2fb7fd9c90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:06.944574 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-74874545-5d76-4b3a-9a71-cc2fb7fd9c90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:07.012463 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1007/2000] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:13:07.014872 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99583b3f-821f-4a07-b86a-6728844c9ff6 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:07.015423 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99583b3f-821f-4a07-b86a-6728844c9ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:07.015906 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99583b3f-821f-4a07-b86a-6728844c9ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:07.059220 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-99583b3f-821f-4a07-b86a-6728844c9ff6 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:07.061227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-99583b3f-821f-4a07-b86a-6728844c9ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:07.075057 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1008/2001] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:07.078159 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99583b3f-821f-4a07-b86a-6728844c9ff6 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:07.081285 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-99583b3f-821f-4a07-b86a-6728844c9ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:07.109487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99583b3f-821f-4a07-b86a-6728844c9ff6 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:07.114250 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74874545-5d76-4b3a-9a71-cc2fb7fd9c90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:07.117454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74874545-5d76-4b3a-9a71-cc2fb7fd9c90 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:07.118677 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e5f2e-c1b9-4a5e-9955-1e57c0b624bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:07.120410 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e5f2e-c1b9-4a5e-9955-1e57c0b624bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:07.121624 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e5f2e-c1b9-4a5e-9955-1e57c0b624bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:07.122192 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e5f2e-c1b9-4a5e-9955-1e57c0b624bb admin admin] PATH_INFO: `/v3/users/d8b5bac3d35c414eaaa10b6a91d8d91d` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:07.124682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e5f2e-c1b9-4a5e-9955-1e57c0b624bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8b5bac3d35c414eaaa10b6a91d8d91d)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:07.129290 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e5f2e-c1b9-4a5e-9955-1e57c0b624bb admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:07.138827 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 994/2002] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:07.140246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74874545-5d76-4b3a-9a71-cc2fb7fd9c90 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:07.158974 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74874545-5d76-4b3a-9a71-cc2fb7fd9c90 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:07.175046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d5dd72-642d-4e73-a854-a80b152643c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:07.177174 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5dd72-642d-4e73-a854-a80b152643c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:07.178130 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5dd72-642d-4e73-a854-a80b152643c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:07.179535 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5dd72-642d-4e73-a854-a80b152643c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:07.180795 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d5dd72-642d-4e73-a854-a80b152643c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:13:07.182306 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d5dd72-642d-4e73-a854-a80b152643c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:07.185788 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d5dd72-642d-4e73-a854-a80b152643c7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:07.210690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 995/2003] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:13:07.231677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d5dd72-642d-4e73-a854-a80b152643c7 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:07.251004 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:07.256134 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d5dd72-642d-4e73-a854-a80b152643c7 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:07.262592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7c9e5f2e-c1b9-4a5e-9955-1e57c0b624bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8b5bac3d35c414eaaa10b6a91d8d91d'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:07.279302 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d733c747-b469-4eb1-9954-807e3e4999d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:07.281023 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733c747-b469-4eb1-9954-807e3e4999d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:07.282779 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733c747-b469-4eb1-9954-807e3e4999d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:07.283497 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733c747-b469-4eb1-9954-807e3e4999d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:07.285018 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d733c747-b469-4eb1-9954-807e3e4999d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:07.288224 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7c9e5f2e-c1b9-4a5e-9955-1e57c0b624bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8b5bac3d35c414eaaa10b6a91d8d91d'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:07.290309 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d733c747-b469-4eb1-9954-807e3e4999d5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:07.358176 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 996/2004] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:07 2022] POST /identity/v3/projects => generated 373 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:13:07.371261 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7c9e5f2e-c1b9-4a5e-9955-1e57c0b624bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8b5bac3d35c414eaaa10b6a91d8d91d'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:07.373067 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7c9e5f2e-c1b9-4a5e-9955-1e57c0b624bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8b5bac3d35c414eaaa10b6a91d8d91d'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:07.376416 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d733c747-b469-4eb1-9954-807e3e4999d5 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:07.400624 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7c9e5f2e-c1b9-4a5e-9955-1e57c0b624bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8b5bac3d35c414eaaa10b6a91d8d91d'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:07.433767 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1009/2005] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:07 2022] DELETE /identity/v3/users/d8b5bac3d35c414eaaa10b6a91d8d91d => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:07.446002 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb05d95-0edb-49d7-aea5-c0bf5f1a52ad None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:07.447058 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb05d95-0edb-49d7-aea5-c0bf5f1a52ad None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:07.447746 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb05d95-0edb-49d7-aea5-c0bf5f1a52ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:07.485913 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-9bb05d95-0edb-49d7-aea5-c0bf5f1a52ad None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:07.487471 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-9bb05d95-0edb-49d7-aea5-c0bf5f1a52ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:07.585655 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d733c747-b469-4eb1-9954-807e3e4999d5 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:07.680808 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1010/2006] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:13:07.698298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb05d95-0edb-49d7-aea5-c0bf5f1a52ad None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:07.703909 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-9bb05d95-0edb-49d7-aea5-c0bf5f1a52ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:07.739835 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:07.785679 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34755e-6846-44a3-9551-8bbbc37434e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:07.787740 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34755e-6846-44a3-9551-8bbbc37434e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:07.788646 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34755e-6846-44a3-9551-8bbbc37434e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:07.789408 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34755e-6846-44a3-9551-8bbbc37434e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:07.790950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34755e-6846-44a3-9551-8bbbc37434e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:07.811403 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34755e-6846-44a3-9551-8bbbc37434e5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:07.856637 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 997/2007] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:07 2022] POST /identity/v3/users => generated 287 bytes in 486 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:07.857882 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34755e-6846-44a3-9551-8bbbc37434e5 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:07.860898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ba34755e-6846-44a3-9551-8bbbc37434e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:07.900488 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34755e-6846-44a3-9551-8bbbc37434e5 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:07.919431 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9dbf4-a6ba-494e-8fa2-d61576561d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:07.921426 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9dbf4-a6ba-494e-8fa2-d61576561d18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:07.923872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9dbf4-a6ba-494e-8fa2-d61576561d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:07.924662 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9dbf4-a6ba-494e-8fa2-d61576561d18 admin admin] PATH_INFO: `/v3/projects/4a1c4db251e347f79bc75af8eef192c1` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:07.927398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb05d95-0edb-49d7-aea5-c0bf5f1a52ad None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:07.931608 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9dbf4-a6ba-494e-8fa2-d61576561d18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a1c4db251e347f79bc75af8eef192c1)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:07.941806 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9dbf4-a6ba-494e-8fa2-d61576561d18 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:07.943443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe0e88-ba08-4fa1-8025-0152044ef230 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:07.946431 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe0e88-ba08-4fa1-8025-0152044ef230 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:07.946958 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe0e88-ba08-4fa1-8025-0152044ef230 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:07.947378 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe0e88-ba08-4fa1-8025-0152044ef230 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:07.999643 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe0e88-ba08-4fa1-8025-0152044ef230 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:08.002918 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe0e88-ba08-4fa1-8025-0152044ef230 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:08.029785 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1011/2008] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:08.031492 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe0e88-ba08-4fa1-8025-0152044ef230 service neutron] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:08.034545 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-3ffe0e88-ba08-4fa1-8025-0152044ef230 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:08.068277 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe0e88-ba08-4fa1-8025-0152044ef230 service neutron] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:08.128662 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-edd9dbf4-a6ba-494e-8fa2-d61576561d18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a1c4db251e347f79bc75af8eef192c1', 'request_id': 'req-edd9dbf4-a6ba-494e-8fa2-d61576561d18'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:08.145429 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-edd9dbf4-a6ba-494e-8fa2-d61576561d18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a1c4db251e347f79bc75af8eef192c1', 'request_id': 'req-edd9dbf4-a6ba-494e-8fa2-d61576561d18'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:08.175469 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-edd9dbf4-a6ba-494e-8fa2-d61576561d18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a1c4db251e347f79bc75af8eef192c1', 'request_id': 'req-edd9dbf4-a6ba-494e-8fa2-d61576561d18'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:08.199359 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-edd9dbf4-a6ba-494e-8fa2-d61576561d18 admin admin] The token cache is being invalidate because project 4a1c4db251e347f79bc75af8eef192c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:08.199907 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-edd9dbf4-a6ba-494e-8fa2-d61576561d18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:08.200522 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-edd9dbf4-a6ba-494e-8fa2-d61576561d18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:08.202855 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 998/2009] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:07 2022] DELETE /identity/v3/projects/4a1c4db251e347f79bc75af8eef192c1 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:08.209067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9dbf4-a6ba-494e-8fa2-d61576561d18 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:08.213713 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-edd9dbf4-a6ba-494e-8fa2-d61576561d18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:08.225545 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb9834-dcd3-45eb-aa2d-b0404d754fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:08.226418 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb9834-dcd3-45eb-aa2d-b0404d754fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:08.226714 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb9834-dcd3-45eb-aa2d-b0404d754fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:08.226991 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb9834-dcd3-45eb-aa2d-b0404d754fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:08.227739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb9834-dcd3-45eb-aa2d-b0404d754fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:08.229599 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb9834-dcd3-45eb-aa2d-b0404d754fb8 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:08.236668 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:08.254193 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1012/2010] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:08.260049 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb9834-dcd3-45eb-aa2d-b0404d754fb8 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:08.263488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-73fb9834-dcd3-45eb-aa2d-b0404d754fb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:08.299072 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb9834-dcd3-45eb-aa2d-b0404d754fb8 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:08.309124 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1870b1-2f7f-415c-8f8f-72fbbc9e4ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:08.310174 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1870b1-2f7f-415c-8f8f-72fbbc9e4ff9 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:08.310475 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1870b1-2f7f-415c-8f8f-72fbbc9e4ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:08.310757 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1870b1-2f7f-415c-8f8f-72fbbc9e4ff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:08.311619 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1870b1-2f7f-415c-8f8f-72fbbc9e4ff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:08.313295 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1870b1-2f7f-415c-8f8f-72fbbc9e4ff9 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:08.330019 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1013/2011] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:13:08.340606 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1870b1-2f7f-415c-8f8f-72fbbc9e4ff9 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:08.363765 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1870b1-2f7f-415c-8f8f-72fbbc9e4ff9 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:08.373327 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249bf238-992f-40ba-bfc8-7ce09463662b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:08.374684 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bf238-992f-40ba-bfc8-7ce09463662b admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:08.375279 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bf238-992f-40ba-bfc8-7ce09463662b admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:08.375406 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9dbf4-a6ba-494e-8fa2-d61576561d18 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:08.375626 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bf238-992f-40ba-bfc8-7ce09463662b admin admin] PATH_INFO: `/v3/projects/95e2f6f268c045f9a262d7333ee5d6eb/users/9785a45283f74ee4b182f1cc7cf1df9b/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:08.382275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae89651-bfcd-45a3-a6a7-62a433cff55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:08.383627 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae89651-bfcd-45a3-a6a7-62a433cff55a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:08.384160 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae89651-bfcd-45a3-a6a7-62a433cff55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:08.385009 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae89651-bfcd-45a3-a6a7-62a433cff55a admin admin] PATH_INFO: `/v3/users/8f71730d94dc47b68154b4d6f6f51ba1` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:08.386521 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae89651-bfcd-45a3-a6a7-62a433cff55a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f71730d94dc47b68154b4d6f6f51ba1)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:08.391435 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae89651-bfcd-45a3-a6a7-62a433cff55a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:08.407253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249bf238-992f-40ba-bfc8-7ce09463662b admin admin] RBAC: Authorizing `identity:create_grant(project_id=95e2f6f268c045f9a262d7333ee5d6eb, user_id=9785a45283f74ee4b182f1cc7cf1df9b, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:08.412177 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249bf238-992f-40ba-bfc8-7ce09463662b admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:08.453965 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1014/2012] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:08 2022] PUT /identity/v3/projects/95e2f6f268c045f9a262d7333ee5d6eb/users/9785a45283f74ee4b182f1cc7cf1df9b/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:08.461658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838d53c-8100-43ef-9d19-13ea3ce51995 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:08.462010 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838d53c-8100-43ef-9d19-13ea3ce51995 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:08.462287 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838d53c-8100-43ef-9d19-13ea3ce51995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:08.483827 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4ae89651-bfcd-45a3-a6a7-62a433cff55a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f71730d94dc47b68154b4d6f6f51ba1'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:08.516129 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=80,keystone:INSERT=4,keystone:DELETE=16 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:08.518991 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4ae89651-bfcd-45a3-a6a7-62a433cff55a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f71730d94dc47b68154b4d6f6f51ba1'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:08.538906 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4ae89651-bfcd-45a3-a6a7-62a433cff55a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f71730d94dc47b68154b4d6f6f51ba1'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:08.555867 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4ae89651-bfcd-45a3-a6a7-62a433cff55a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f71730d94dc47b68154b4d6f6f51ba1'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:08.558942 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4ae89651-bfcd-45a3-a6a7-62a433cff55a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f71730d94dc47b68154b4d6f6f51ba1'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:08.566028 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 999/2013] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:08 2022] DELETE /identity/v3/users/8f71730d94dc47b68154b4d6f6f51ba1 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:08.579337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-c838d53c-8100-43ef-9d19-13ea3ce51995 None None] MFA Rules not processed for user `9785a45283f74ee4b182f1cc7cf1df9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:08.593795 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-c838d53c-8100-43ef-9d19-13ea3ce51995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:08.612203 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae89651-bfcd-45a3-a6a7-62a433cff55a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:08.613856 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4ae89651-bfcd-45a3-a6a7-62a433cff55a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:08.736366 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1015/2014] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:08.740233 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae89651-bfcd-45a3-a6a7-62a433cff55a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:08.755915 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838d53c-8100-43ef-9d19-13ea3ce51995 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:08.760474 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-c838d53c-8100-43ef-9d19-13ea3ce51995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:08.762104 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e95dea-edd4-43ca-89f8-4928c5776151 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:08.763675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e95dea-edd4-43ca-89f8-4928c5776151 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:08.764511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e95dea-edd4-43ca-89f8-4928c5776151 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:08.765294 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e95dea-edd4-43ca-89f8-4928c5776151 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:08.769418 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a4e95dea-edd4-43ca-89f8-4928c5776151 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:08.789713 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838d53c-8100-43ef-9d19-13ea3ce51995 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:08.796734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823d3b3-ef45-4b36-9ecd-82832b22fa7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:08.798667 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823d3b3-ef45-4b36-9ecd-82832b22fa7c service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:08.799480 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823d3b3-ef45-4b36-9ecd-82832b22fa7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:08.800660 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823d3b3-ef45-4b36-9ecd-82832b22fa7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:08.830659 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823d3b3-ef45-4b36-9ecd-82832b22fa7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:08.834898 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823d3b3-ef45-4b36-9ecd-82832b22fa7c service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:08.853305 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1016/2015] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:08.888958 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823d3b3-ef45-4b36-9ecd-82832b22fa7c service cinder] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:08.892342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-d823d3b3-ef45-4b36-9ecd-82832b22fa7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:08.943084 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e95dea-edd4-43ca-89f8-4928c5776151 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:08.944563 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e95dea-edd4-43ca-89f8-4928c5776151 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:08.970748 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1000/2016] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:08 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:08.973440 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e95dea-edd4-43ca-89f8-4928c5776151 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:08.977709 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a4e95dea-edd4-43ca-89f8-4928c5776151 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:09.121806 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823d3b3-ef45-4b36-9ecd-82832b22fa7c service cinder] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:09.136611 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc2825-37d0-4daa-82f7-46784d5f2a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:09.138776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc2825-37d0-4daa-82f7-46784d5f2a71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:09.139785 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc2825-37d0-4daa-82f7-46784d5f2a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:09.140227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc2825-37d0-4daa-82f7-46784d5f2a71 admin admin] PATH_INFO: `/v3/projects/96e729e07876487bac9e8016467350a3` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:09.142912 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bc2825-37d0-4daa-82f7-46784d5f2a71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96e729e07876487bac9e8016467350a3)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:09.147480 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bc2825-37d0-4daa-82f7-46784d5f2a71 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:09.186918 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e95dea-edd4-43ca-89f8-4928c5776151 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:09.202392 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebaa98a-8df9-4cd5-a327-2a106f140a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:09.204717 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebaa98a-8df9-4cd5-a327-2a106f140a24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:09.205066 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebaa98a-8df9-4cd5-a327-2a106f140a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:09.210210 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebaa98a-8df9-4cd5-a327-2a106f140a24 admin admin] PATH_INFO: `/v3/users/9785a45283f74ee4b182f1cc7cf1df9b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:09.212997 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebaa98a-8df9-4cd5-a327-2a106f140a24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9785a45283f74ee4b182f1cc7cf1df9b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:09.217853 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebaa98a-8df9-4cd5-a327-2a106f140a24 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:09.270767 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:09.283791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-12bc2825-37d0-4daa-82f7-46784d5f2a71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96e729e07876487bac9e8016467350a3', 'request_id': 'req-12bc2825-37d0-4daa-82f7-46784d5f2a71'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:09.308515 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-12bc2825-37d0-4daa-82f7-46784d5f2a71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96e729e07876487bac9e8016467350a3', 'request_id': 'req-12bc2825-37d0-4daa-82f7-46784d5f2a71'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:09.323280 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:09.327129 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-12bc2825-37d0-4daa-82f7-46784d5f2a71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96e729e07876487bac9e8016467350a3', 'request_id': 'req-12bc2825-37d0-4daa-82f7-46784d5f2a71'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:09.371662 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-12bc2825-37d0-4daa-82f7-46784d5f2a71 admin admin] The token cache is being invalidate because project 96e729e07876487bac9e8016467350a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:09.372932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-12bc2825-37d0-4daa-82f7-46784d5f2a71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:09.374747 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-12bc2825-37d0-4daa-82f7-46784d5f2a71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:09.375643 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1ebaa98a-8df9-4cd5-a327-2a106f140a24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9785a45283f74ee4b182f1cc7cf1df9b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:09.386072 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1017/2017] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:08 2022] DELETE /identity/v3/projects/96e729e07876487bac9e8016467350a3 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:09.392471 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa72c1-105d-4461-9f8c-ee011b213535 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:09.392981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa72c1-105d-4461-9f8c-ee011b213535 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:09.393541 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa72c1-105d-4461-9f8c-ee011b213535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:09.416740 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1ebaa98a-8df9-4cd5-a327-2a106f140a24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9785a45283f74ee4b182f1cc7cf1df9b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:09.439657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1ebaa98a-8df9-4cd5-a327-2a106f140a24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9785a45283f74ee4b182f1cc7cf1df9b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:09.466052 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-6dfa72c1-105d-4461-9f8c-ee011b213535 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:09.466052 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-6dfa72c1-105d-4461-9f8c-ee011b213535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:09.477148 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1ebaa98a-8df9-4cd5-a327-2a106f140a24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9785a45283f74ee4b182f1cc7cf1df9b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:09.478896 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1ebaa98a-8df9-4cd5-a327-2a106f140a24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9785a45283f74ee4b182f1cc7cf1df9b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:09.488737 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1001/2018] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:08 2022] DELETE /identity/v3/users/9785a45283f74ee4b182f1cc7cf1df9b => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:09.495024 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a8a78-25e1-449e-aad4-6695bfd884af None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:09.496198 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a8a78-25e1-449e-aad4-6695bfd884af None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:09.496783 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a8a78-25e1-449e-aad4-6695bfd884af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:09.546066 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-947a8a78-25e1-449e-aad4-6695bfd884af None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:09.547956 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-947a8a78-25e1-449e-aad4-6695bfd884af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:09.652137 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1018/2019] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:09.652902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586de659-c55f-4eb0-8505-46226d54efb4 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:09.653353 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586de659-c55f-4eb0-8505-46226d54efb4 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:09.653798 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586de659-c55f-4eb0-8505-46226d54efb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:09.689275 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-586de659-c55f-4eb0-8505-46226d54efb4 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:09.692882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-586de659-c55f-4eb0-8505-46226d54efb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:09.720314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1002/2020] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:13:09.755954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a8a78-25e1-449e-aad4-6695bfd884af None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:09.763867 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-947a8a78-25e1-449e-aad4-6695bfd884af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:09.845685 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1019/2021] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:09.854069 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586de659-c55f-4eb0-8505-46226d54efb4 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:09.876572 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586de659-c55f-4eb0-8505-46226d54efb4 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:09.885278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4a1a2-26d8-4204-8d38-880b08af4aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:09.886922 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4a1a2-26d8-4204-8d38-880b08af4aea admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:09.887301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4a1a2-26d8-4204-8d38-880b08af4aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:09.888407 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4a1a2-26d8-4204-8d38-880b08af4aea admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:09.889536 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc4a1a2-26d8-4204-8d38-880b08af4aea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:13:09.890152 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc4a1a2-26d8-4204-8d38-880b08af4aea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:09.895992 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc4a1a2-26d8-4204-8d38-880b08af4aea admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:09.912087 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a8a78-25e1-449e-aad4-6695bfd884af None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:09.918022 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b80cc-e71d-467c-b396-e40898d38eff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:09.919051 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b80cc-e71d-467c-b396-e40898d38eff service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:09.919373 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b80cc-e71d-467c-b396-e40898d38eff service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:09.919862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b80cc-e71d-467c-b396-e40898d38eff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:09.925170 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1020/2022] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:13:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:13:09.926761 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4a1a2-26d8-4204-8d38-880b08af4aea admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:09.941591 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b80cc-e71d-467c-b396-e40898d38eff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:09.944843 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b80cc-e71d-467c-b396-e40898d38eff service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:09.960670 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1003/2023] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:09.960972 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4a1a2-26d8-4204-8d38-880b08af4aea admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:09.961685 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b80cc-e71d-467c-b396-e40898d38eff service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:09.964048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-734b80cc-e71d-467c-b396-e40898d38eff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:09.982184 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea6b99-f9f2-4f62-ab67-50cf0a99e52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:09.982493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b80cc-e71d-467c-b396-e40898d38eff service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:09.984128 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea6b99-f9f2-4f62-ab67-50cf0a99e52f admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:09.984880 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea6b99-f9f2-4f62-ab67-50cf0a99e52f admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:09.985612 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea6b99-f9f2-4f62-ab67-50cf0a99e52f admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:09.987525 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea6b99-f9f2-4f62-ab67-50cf0a99e52f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:09.994125 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea6b99-f9f2-4f62-ab67-50cf0a99e52f admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:10.000729 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce317c-62ad-44fa-b7cf-498b82a43507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:10.002970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce317c-62ad-44fa-b7cf-498b82a43507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:10.003636 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce317c-62ad-44fa-b7cf-498b82a43507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:10.004195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce317c-62ad-44fa-b7cf-498b82a43507 admin admin] PATH_INFO: `/v3/projects/d904faabfae04291b048160634586b66` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:10.007253 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ce317c-62ad-44fa-b7cf-498b82a43507 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d904faabfae04291b048160634586b66)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:10.010052 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ce317c-62ad-44fa-b7cf-498b82a43507 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:10.054002 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1021/2024] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:09 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:10.061061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea6b99-f9f2-4f62-ab67-50cf0a99e52f admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:10.111701 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:10.114367 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-73ce317c-62ad-44fa-b7cf-498b82a43507 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd904faabfae04291b048160634586b66', 'request_id': 'req-73ce317c-62ad-44fa-b7cf-498b82a43507'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:10.131222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-73ce317c-62ad-44fa-b7cf-498b82a43507 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd904faabfae04291b048160634586b66', 'request_id': 'req-73ce317c-62ad-44fa-b7cf-498b82a43507'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:10.150590 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-73ce317c-62ad-44fa-b7cf-498b82a43507 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd904faabfae04291b048160634586b66', 'request_id': 'req-73ce317c-62ad-44fa-b7cf-498b82a43507'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:10.190812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-73ce317c-62ad-44fa-b7cf-498b82a43507 admin admin] The token cache is being invalidate because project d904faabfae04291b048160634586b66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:10.191455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-73ce317c-62ad-44fa-b7cf-498b82a43507 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:10.192110 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-73ce317c-62ad-44fa-b7cf-498b82a43507 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:10.194087 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1004/2025] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:09 2022] DELETE /identity/v3/projects/d904faabfae04291b048160634586b66 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:10.202811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a678f-f242-4360-878f-148f56366899 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:10.203321 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a678f-f242-4360-878f-148f56366899 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:10.203832 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a678f-f242-4360-878f-148f56366899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:10.204529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:10.259752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-747a678f-f242-4360-878f-148f56366899 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:10.263265 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-747a678f-f242-4360-878f-148f56366899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:10.265229 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea6b99-f9f2-4f62-ab67-50cf0a99e52f admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:10.392062 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e066756-d12e-4fd2-9ede-baf1761ac8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:10.394337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e066756-d12e-4fd2-9ede-baf1761ac8c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:10.394337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e066756-d12e-4fd2-9ede-baf1761ac8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:10.394896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e066756-d12e-4fd2-9ede-baf1761ac8c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:10.395625 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e066756-d12e-4fd2-9ede-baf1761ac8c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:10.398598 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e066756-d12e-4fd2-9ede-baf1761ac8c4 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:10.417902 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1005/2026] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:13:10.428091 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e89698-9fa3-4b4f-a865-de81f7175951 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:10.428563 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e89698-9fa3-4b4f-a865-de81f7175951 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:10.428812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e89698-9fa3-4b4f-a865-de81f7175951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:10.448155 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1022/2027] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:10 2022] POST /identity/v3/users => generated 293 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:13:10.451388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e066756-d12e-4fd2-9ede-baf1761ac8c4 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:10.452999 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-1e066756-d12e-4fd2-9ede-baf1761ac8c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:10.461011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f6e89698-9fa3-4b4f-a865-de81f7175951 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:10.464764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f6e89698-9fa3-4b4f-a865-de81f7175951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:10.476828 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e066756-d12e-4fd2-9ede-baf1761ac8c4 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:10.483287 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cc46e-eaa5-48d9-aa49-611ce4cf93a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:10.485008 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cc46e-eaa5-48d9-aa49-611ce4cf93a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:10.485008 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cc46e-eaa5-48d9-aa49-611ce4cf93a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:10.485642 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cc46e-eaa5-48d9-aa49-611ce4cf93a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:10.496443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431cc46e-eaa5-48d9-aa49-611ce4cf93a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:10.499922 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431cc46e-eaa5-48d9-aa49-611ce4cf93a2 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:10.503048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1006/2028] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:10.504428 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e89698-9fa3-4b4f-a865-de81f7175951 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:10.533411 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1023/2029] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:13:10.535320 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cc46e-eaa5-48d9-aa49-611ce4cf93a2 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:10.536581 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e89698-9fa3-4b4f-a865-de81f7175951 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:10.569385 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cc46e-eaa5-48d9-aa49-611ce4cf93a2 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:10.573426 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d64aa9-a111-4c6a-93a3-297101321729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:10.574884 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d64aa9-a111-4c6a-93a3-297101321729 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:10.578589 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d64aa9-a111-4c6a-93a3-297101321729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:10.579061 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d64aa9-a111-4c6a-93a3-297101321729 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:10.580140 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d64aa9-a111-4c6a-93a3-297101321729 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:13:10.580562 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d64aa9-a111-4c6a-93a3-297101321729 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:10.583133 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d64aa9-a111-4c6a-93a3-297101321729 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:10.584573 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1085b6f-bc25-4939-a889-543b10e7bae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:10.585707 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1085b6f-bc25-4939-a889-543b10e7bae3 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:10.587688 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1085b6f-bc25-4939-a889-543b10e7bae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:10.588754 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1085b6f-bc25-4939-a889-543b10e7bae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:10.591042 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1085b6f-bc25-4939-a889-543b10e7bae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:10.597533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1085b6f-bc25-4939-a889-543b10e7bae3 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:10.619447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1007/2030] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:13:10.625480 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1024/2031] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:10.628182 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d64aa9-a111-4c6a-93a3-297101321729 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:10.629451 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1085b6f-bc25-4939-a889-543b10e7bae3 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:10.632209 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-c1085b6f-bc25-4939-a889-543b10e7bae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:10.653375 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d64aa9-a111-4c6a-93a3-297101321729 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:10.655234 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1085b6f-bc25-4939-a889-543b10e7bae3 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:10.663202 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01f678-1f3d-40ad-83ef-3ca64cd34a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:10.664518 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe6213-038e-45b8-829d-31da3af898ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:10.665891 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe6213-038e-45b8-829d-31da3af898ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:10.666056 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01f678-1f3d-40ad-83ef-3ca64cd34a86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:10.666304 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01f678-1f3d-40ad-83ef-3ca64cd34a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:10.666511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe6213-038e-45b8-829d-31da3af898ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:10.666653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01f678-1f3d-40ad-83ef-3ca64cd34a86 admin admin] PATH_INFO: `/v3/projects/e53005358da24611a3e97ae2b5073b99/users/0ea596fe816948438df6d9648348e6e9/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:10.666977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe6213-038e-45b8-829d-31da3af898ed admin admin] PATH_INFO: `/v3/projects/95e2f6f268c045f9a262d7333ee5d6eb` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:10.670597 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fe6213-038e-45b8-829d-31da3af898ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=95e2f6f268c045f9a262d7333ee5d6eb)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:10.674365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fe6213-038e-45b8-829d-31da3af898ed admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:10.692682 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad01f678-1f3d-40ad-83ef-3ca64cd34a86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e53005358da24611a3e97ae2b5073b99, user_id=0ea596fe816948438df6d9648348e6e9, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:10.698927 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad01f678-1f3d-40ad-83ef-3ca64cd34a86 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:10.725199 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1008/2032] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:10 2022] PUT /identity/v3/projects/e53005358da24611a3e97ae2b5073b99/users/0ea596fe816948438df6d9648348e6e9/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:10.735834 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01f678-1f3d-40ad-83ef-3ca64cd34a86 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:10.834199 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-e7fe6213-038e-45b8-829d-31da3af898ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95e2f6f268c045f9a262d7333ee5d6eb', 'request_id': 'req-e7fe6213-038e-45b8-829d-31da3af898ed'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:10.846470 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-e7fe6213-038e-45b8-829d-31da3af898ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95e2f6f268c045f9a262d7333ee5d6eb', 'request_id': 'req-e7fe6213-038e-45b8-829d-31da3af898ed'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:10.861365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-e7fe6213-038e-45b8-829d-31da3af898ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95e2f6f268c045f9a262d7333ee5d6eb', 'request_id': 'req-e7fe6213-038e-45b8-829d-31da3af898ed'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:10.880456 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-e7fe6213-038e-45b8-829d-31da3af898ed admin admin] The token cache is being invalidate because project 95e2f6f268c045f9a262d7333ee5d6eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:10.880903 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-e7fe6213-038e-45b8-829d-31da3af898ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:10.881709 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-e7fe6213-038e-45b8-829d-31da3af898ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:10.889837 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1025/2033] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:10 2022] DELETE /identity/v3/projects/95e2f6f268c045f9a262d7333ee5d6eb => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:10.892349 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a06ec-4a00-4c18-b71a-e686e36d61d8 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:10.893000 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a06ec-4a00-4c18-b71a-e686e36d61d8 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:10.893647 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a06ec-4a00-4c18-b71a-e686e36d61d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:10.921616 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01f678-1f3d-40ad-83ef-3ca64cd34a86 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:10.943977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-413a06ec-4a00-4c18-b71a-e686e36d61d8 None None] MFA Rules not processed for user `0ea596fe816948438df6d9648348e6e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:10.950790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-413a06ec-4a00-4c18-b71a-e686e36d61d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:11.032305 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:11.078004 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:11.110829 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0804228-4fce-4774-8731-a4f1dfd0de83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:11.114500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0804228-4fce-4774-8731-a4f1dfd0de83 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:11.115225 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0804228-4fce-4774-8731-a4f1dfd0de83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:11.119625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0804228-4fce-4774-8731-a4f1dfd0de83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:11.137818 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0804228-4fce-4774-8731-a4f1dfd0de83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:11.143164 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0804228-4fce-4774-8731-a4f1dfd0de83 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:11.184317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1026/2034] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:11.189443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6026c-e21e-40fb-874c-4bc438bf71f0 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:11.189780 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6026c-e21e-40fb-874c-4bc438bf71f0 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:11.190130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6026c-e21e-40fb-874c-4bc438bf71f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:11.438779 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-80c6026c-e21e-40fb-874c-4bc438bf71f0 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:11.438779 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-80c6026c-e21e-40fb-874c-4bc438bf71f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:11.438779 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1027/2035] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:11.438779 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c6026c-e21e-40fb-874c-4bc438bf71f0 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:11.438779 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-80c6026c-e21e-40fb-874c-4bc438bf71f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:11.440077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c6026c-e21e-40fb-874c-4bc438bf71f0 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:11.440077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5257f4e-67ea-4fdc-9340-8756c3de5ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:11.440077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5257f4e-67ea-4fdc-9340-8756c3de5ea8 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:11.440077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5257f4e-67ea-4fdc-9340-8756c3de5ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:11.440545 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5257f4e-67ea-4fdc-9340-8756c3de5ea8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:11.440545 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5257f4e-67ea-4fdc-9340-8756c3de5ea8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:11.440545 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5257f4e-67ea-4fdc-9340-8756c3de5ea8 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:11.486555 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1009/2036] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:10 2022] POST /identity/v3/projects => generated 369 bytes in 756 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:11.486555 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5e64d-12e1-49f2-bd52-ffb1138b42c6 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:11.486555 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5e64d-12e1-49f2-bd52-ffb1138b42c6 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:11.486555 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5e64d-12e1-49f2-bd52-ffb1138b42c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:11.493013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1028/2037] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:11 2022] POST /identity/v3/projects => generated 383 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:11.493696 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5257f4e-67ea-4fdc-9340-8756c3de5ea8 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:11.500554 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-b5257f4e-67ea-4fdc-9340-8756c3de5ea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:11.517213 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-6cd5e64d-12e1-49f2-bd52-ffb1138b42c6 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:11.519317 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6cd5e64d-12e1-49f2-bd52-ffb1138b42c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:11.677582 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1010/2038] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:11.679982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5e64d-12e1-49f2-bd52-ffb1138b42c6 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:11.698335 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5257f4e-67ea-4fdc-9340-8756c3de5ea8 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:11.710976 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145aa24-2d69-4849-a573-808ebb669382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:11.712574 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5e64d-12e1-49f2-bd52-ffb1138b42c6 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:11.713067 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145aa24-2d69-4849-a573-808ebb669382 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:11.713621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145aa24-2d69-4849-a573-808ebb669382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:11.714866 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145aa24-2d69-4849-a573-808ebb669382 admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:11.716473 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145aa24-2d69-4849-a573-808ebb669382 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:11.720770 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145aa24-2d69-4849-a573-808ebb669382 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:11.721399 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e1c07-9dc5-47b3-92bd-3eef737dfa4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:11.722408 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e1c07-9dc5-47b3-92bd-3eef737dfa4a admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:11.722797 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e1c07-9dc5-47b3-92bd-3eef737dfa4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:11.723195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e1c07-9dc5-47b3-92bd-3eef737dfa4a admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:11.724237 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e1c07-9dc5-47b3-92bd-3eef737dfa4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:11.726567 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e1c07-9dc5-47b3-92bd-3eef737dfa4a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:11.772544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1011/2039] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:11 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:11.774181 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e1c07-9dc5-47b3-92bd-3eef737dfa4a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:11.775002 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1029/2040] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:11 2022] POST /identity/v3/users => generated 285 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:11.778612 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145aa24-2d69-4849-a573-808ebb669382 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:11.790706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145aa24-2d69-4849-a573-808ebb669382 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:11.802031 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e1c07-9dc5-47b3-92bd-3eef737dfa4a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:11.802392 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54742653-b367-4a1a-b825-ea03a4c7705c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:11.803601 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54742653-b367-4a1a-b825-ea03a4c7705c admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:11.804322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54742653-b367-4a1a-b825-ea03a4c7705c admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:11.804940 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54742653-b367-4a1a-b825-ea03a4c7705c admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:11.806233 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54742653-b367-4a1a-b825-ea03a4c7705c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:11.808988 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ed513-e86e-465e-8204-0bd71a9c015d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:11.809816 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ed513-e86e-465e-8204-0bd71a9c015d admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:11.810049 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54742653-b367-4a1a-b825-ea03a4c7705c admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:11.811104 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ed513-e86e-465e-8204-0bd71a9c015d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:11.811104 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ed513-e86e-465e-8204-0bd71a9c015d admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:11.811691 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ed513-e86e-465e-8204-0bd71a9c015d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:13:11.812164 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ed513-e86e-465e-8204-0bd71a9c015d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:11.815673 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ed513-e86e-465e-8204-0bd71a9c015d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:11.833532 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1012/2041] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:13:11.837307 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ed513-e86e-465e-8204-0bd71a9c015d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:11.843683 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1030/2042] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:13:11.845383 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54742653-b367-4a1a-b825-ea03a4c7705c admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:11.861751 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ed513-e86e-465e-8204-0bd71a9c015d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:11.871060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54742653-b367-4a1a-b825-ea03a4c7705c admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:11.883669 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d64ead-eed1-4f0e-a794-ee965f4dcca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:11.884950 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d64ead-eed1-4f0e-a794-ee965f4dcca2 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:11.885459 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d64ead-eed1-4f0e-a794-ee965f4dcca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:11.886649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d64ead-eed1-4f0e-a794-ee965f4dcca2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:11.888218 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d64ead-eed1-4f0e-a794-ee965f4dcca2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:11.891302 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb64c6-19d2-4b91-9930-b21b9c073140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:11.893709 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb64c6-19d2-4b91-9930-b21b9c073140 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:11.894577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb64c6-19d2-4b91-9930-b21b9c073140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:11.894975 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb64c6-19d2-4b91-9930-b21b9c073140 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:11.895824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d64ead-eed1-4f0e-a794-ee965f4dcca2 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:11.896005 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb64c6-19d2-4b91-9930-b21b9c073140 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:11.898869 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb64c6-19d2-4b91-9930-b21b9c073140 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:11.920675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1013/2043] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:11.921898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb64c6-19d2-4b91-9930-b21b9c073140 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:11.935891 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb64c6-19d2-4b91-9930-b21b9c073140 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:11.941973 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54af00b-4248-433e-b9c6-8c7b55fa6fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:11.942907 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54af00b-4248-433e-b9c6-8c7b55fa6fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:11.943333 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54af00b-4248-433e-b9c6-8c7b55fa6fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:11.943857 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54af00b-4248-433e-b9c6-8c7b55fa6fe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:11.944746 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54af00b-4248-433e-b9c6-8c7b55fa6fe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:11.946614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54af00b-4248-433e-b9c6-8c7b55fa6fe4 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:11.950237 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1031/2044] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:11 2022] POST /identity/v3/projects => generated 391 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:11.950237 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d64ead-eed1-4f0e-a794-ee965f4dcca2 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:11.963005 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1014/2045] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:11.964055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54af00b-4248-433e-b9c6-8c7b55fa6fe4 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:11.977288 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54af00b-4248-433e-b9c6-8c7b55fa6fe4 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:12.053493 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:12.069637 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d64ead-eed1-4f0e-a794-ee965f4dcca2 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:12.080192 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a4ee2-4d96-44ad-8154-8d497b89e564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:12.081349 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a4ee2-4d96-44ad-8154-8d497b89e564 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:12.081832 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a4ee2-4d96-44ad-8154-8d497b89e564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:12.082408 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a4ee2-4d96-44ad-8154-8d497b89e564 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:12.084246 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a4ee2-4d96-44ad-8154-8d497b89e564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:12.086942 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a4ee2-4d96-44ad-8154-8d497b89e564 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:12.098472 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94c220-8d85-4c80-a459-686a5247ae7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:12.098472 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94c220-8d85-4c80-a459-686a5247ae7b admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:12.098472 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94c220-8d85-4c80-a459-686a5247ae7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:12.099218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94c220-8d85-4c80-a459-686a5247ae7b admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:12.099218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db94c220-8d85-4c80-a459-686a5247ae7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:12.101230 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db94c220-8d85-4c80-a459-686a5247ae7b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:12.114036 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1032/2046] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:12.114036 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a4ee2-4d96-44ad-8154-8d497b89e564 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:12.129359 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1015/2047] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:11 2022] POST /identity/v3/users => generated 296 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:12.130105 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94c220-8d85-4c80-a459-686a5247ae7b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:12.143416 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94c220-8d85-4c80-a459-686a5247ae7b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:12.144524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a4ee2-4d96-44ad-8154-8d497b89e564 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:12.144663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:12.155339 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc5691-cf4d-4445-8356-32a6ce229895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:12.156760 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414b937-2065-4a38-8308-822487f54917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:12.156996 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc5691-cf4d-4445-8356-32a6ce229895 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:12.157353 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc5691-cf4d-4445-8356-32a6ce229895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:12.157762 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc5691-cf4d-4445-8356-32a6ce229895 admin admin] PATH_INFO: `/v3/projects/7b72cdf8559f4a949fa568b877ea1080/users/eabca44411d04808a4c8862728b226e0/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:12.158827 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414b937-2065-4a38-8308-822487f54917 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:12.159365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414b937-2065-4a38-8308-822487f54917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:12.159662 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414b937-2065-4a38-8308-822487f54917 admin admin] PATH_INFO: `/v3/projects/b6fb159380c44bba94ad1745856c1697/users/1096eecae1fb4d318da932e2b440a4f4/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:12.190725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcc5691-cf4d-4445-8356-32a6ce229895 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b72cdf8559f4a949fa568b877ea1080, user_id=eabca44411d04808a4c8862728b226e0, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:12.191183 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d414b937-2065-4a38-8308-822487f54917 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6fb159380c44bba94ad1745856c1697, user_id=1096eecae1fb4d318da932e2b440a4f4, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:12.195760 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d414b937-2065-4a38-8308-822487f54917 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:12.198412 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcc5691-cf4d-4445-8356-32a6ce229895 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:12.219581 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1033/2048] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:12 2022] PUT /identity/v3/projects/b6fb159380c44bba94ad1745856c1697/users/1096eecae1fb4d318da932e2b440a4f4/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:12.221481 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414b937-2065-4a38-8308-822487f54917 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:12.250998 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1016/2049] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:12 2022] PUT /identity/v3/projects/7b72cdf8559f4a949fa568b877ea1080/users/eabca44411d04808a4c8862728b226e0/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:12.250998 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19ff73-c211-4ef6-a7ed-b5d3af2fd96d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:12.250998 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19ff73-c211-4ef6-a7ed-b5d3af2fd96d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:12.251783 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19ff73-c211-4ef6-a7ed-b5d3af2fd96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:12.315018 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-5e19ff73-c211-4ef6-a7ed-b5d3af2fd96d None None] MFA Rules not processed for user `1096eecae1fb4d318da932e2b440a4f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:12.317362 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5e19ff73-c211-4ef6-a7ed-b5d3af2fd96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:12.376635 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414b937-2065-4a38-8308-822487f54917 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:12.503928 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b38edd-2d19-4acd-bcf2-770cc529ceaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:12.505459 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b38edd-2d19-4acd-bcf2-770cc529ceaf admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:12.505903 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b38edd-2d19-4acd-bcf2-770cc529ceaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:12.506300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1017/2050] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:12.513123 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b38edd-2d19-4acd-bcf2-770cc529ceaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:12.513123 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b38edd-2d19-4acd-bcf2-770cc529ceaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:12.513369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebbce6-3658-480e-98ef-28a5392c5b2a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:12.513369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebbce6-3658-480e-98ef-28a5392c5b2a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:12.513628 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b38edd-2d19-4acd-bcf2-770cc529ceaf admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:12.513772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebbce6-3658-480e-98ef-28a5392c5b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:12.554702 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1034/2051] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:12.555619 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b38edd-2d19-4acd-bcf2-770cc529ceaf admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:12.617035 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-59ebbce6-3658-480e-98ef-28a5392c5b2a None None] MFA Rules not processed for user `eabca44411d04808a4c8862728b226e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:12.624917 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-59ebbce6-3658-480e-98ef-28a5392c5b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:12.628262 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b38edd-2d19-4acd-bcf2-770cc529ceaf admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:12.657562 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc58ef7-5a3b-47f6-86b4-4419bc71fcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:12.659583 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc58ef7-5a3b-47f6-86b4-4419bc71fcf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:12.662246 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc58ef7-5a3b-47f6-86b4-4419bc71fcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:12.662960 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc58ef7-5a3b-47f6-86b4-4419bc71fcf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:12.664380 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc58ef7-5a3b-47f6-86b4-4419bc71fcf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:12.675158 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc58ef7-5a3b-47f6-86b4-4419bc71fcf3 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:12.760728 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1035/2052] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:12 2022] POST /identity/v3/projects => generated 385 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:12.764895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc58ef7-5a3b-47f6-86b4-4419bc71fcf3 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:12.876191 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1018/2053] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:12 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:12.877230 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ebbce6-3658-480e-98ef-28a5392c5b2a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:12.898521 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ebbce6-3658-480e-98ef-28a5392c5b2a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:12.906556 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:12.934538 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc58ef7-5a3b-47f6-86b4-4419bc71fcf3 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:12.941785 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa19ea-894c-45ba-a0df-cbdb91b5dcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:12.941785 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa19ea-894c-45ba-a0df-cbdb91b5dcfb admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:12.942518 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa19ea-894c-45ba-a0df-cbdb91b5dcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:12.942518 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa19ea-894c-45ba-a0df-cbdb91b5dcfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:12.943126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa19ea-894c-45ba-a0df-cbdb91b5dcfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:12.945208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa19ea-894c-45ba-a0df-cbdb91b5dcfb admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:12.962627 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1036/2054] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:12.964214 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa19ea-894c-45ba-a0df-cbdb91b5dcfb admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:12.986000 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa19ea-894c-45ba-a0df-cbdb91b5dcfb admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:13.003390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb89a0a-c0bb-41af-be04-077c15b3fe30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:13.003672 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb89a0a-c0bb-41af-be04-077c15b3fe30 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:13.003960 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb89a0a-c0bb-41af-be04-077c15b3fe30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:13.004434 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb89a0a-c0bb-41af-be04-077c15b3fe30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:13.009782 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb89a0a-c0bb-41af-be04-077c15b3fe30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:13.012438 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb89a0a-c0bb-41af-be04-077c15b3fe30 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:13.029718 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:13.044490 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8763b5c4-ded2-4cb6-ae93-b591d2cd45c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:13.045300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763b5c4-ded2-4cb6-ae93-b591d2cd45c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:13.045590 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763b5c4-ded2-4cb6-ae93-b591d2cd45c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:13.046658 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763b5c4-ded2-4cb6-ae93-b591d2cd45c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:13.046870 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8763b5c4-ded2-4cb6-ae93-b591d2cd45c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:13.049192 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8763b5c4-ded2-4cb6-ae93-b591d2cd45c4 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:13.103062 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1019/2055] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:12 2022] POST /identity/v3/users => generated 299 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:13.104551 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8763b5c4-ded2-4cb6-ae93-b591d2cd45c4 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:13.114379 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1037/2056] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:12 2022] POST /identity/v3/projects => generated 369 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:13.114379 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb89a0a-c0bb-41af-be04-077c15b3fe30 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:13.129986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8763b5c4-ded2-4cb6-ae93-b591d2cd45c4 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:13.262171 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb89a0a-c0bb-41af-be04-077c15b3fe30 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:13.274119 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbe939-fc31-4b71-af6b-4b8e619c2f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:13.275026 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbe939-fc31-4b71-af6b-4b8e619c2f64 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:13.275401 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbe939-fc31-4b71-af6b-4b8e619c2f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:13.275822 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbe939-fc31-4b71-af6b-4b8e619c2f64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:13.278610 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccbe939-fc31-4b71-af6b-4b8e619c2f64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:13.287116 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccbe939-fc31-4b71-af6b-4b8e619c2f64 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:13.304953 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1038/2057] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:13.306348 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbe939-fc31-4b71-af6b-4b8e619c2f64 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:13.322762 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbe939-fc31-4b71-af6b-4b8e619c2f64 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:13.330870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839af48-03c8-48e8-a8ac-d49c12dd4186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:13.331998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839af48-03c8-48e8-a8ac-d49c12dd4186 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:13.332384 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839af48-03c8-48e8-a8ac-d49c12dd4186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:13.332768 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839af48-03c8-48e8-a8ac-d49c12dd4186 admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:13.333671 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4839af48-03c8-48e8-a8ac-d49c12dd4186 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:13.336701 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4839af48-03c8-48e8-a8ac-d49c12dd4186 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:13.342985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2c65d-2914-4c21-8c11-531acfd49998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:13.344932 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2c65d-2914-4c21-8c11-531acfd49998 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:13.345567 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2c65d-2914-4c21-8c11-531acfd49998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:13.346400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2c65d-2914-4c21-8c11-531acfd49998 admin admin] PATH_INFO: `/v3/projects/29303a4700e340c6a90f0233af0fbfac/users/136d02137cb8458ab66659512fd11f2a/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:13.373907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1039/2058] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:13 2022] POST /identity/v3/users => generated 291 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:13.373907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839af48-03c8-48e8-a8ac-d49c12dd4186 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:13.380079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2c65d-2914-4c21-8c11-531acfd49998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29303a4700e340c6a90f0233af0fbfac, user_id=136d02137cb8458ab66659512fd11f2a, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:13.389118 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2c65d-2914-4c21-8c11-531acfd49998 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:13.394647 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839af48-03c8-48e8-a8ac-d49c12dd4186 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:13.404028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24564e7e-f526-42ce-b3a9-08574ba5b338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:13.405034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24564e7e-f526-42ce-b3a9-08574ba5b338 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:13.405471 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24564e7e-f526-42ce-b3a9-08574ba5b338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:13.405866 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24564e7e-f526-42ce-b3a9-08574ba5b338 admin admin] PATH_INFO: `/v3/projects/dc7f9b96acb14aca890e684ef19028ba/users/25371fa9f9264032bc796aed558ff250/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:13.422076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1020/2059] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:13 2022] PUT /identity/v3/projects/29303a4700e340c6a90f0233af0fbfac/users/136d02137cb8458ab66659512fd11f2a/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:13.424108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2c65d-2914-4c21-8c11-531acfd49998 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:13.441443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24564e7e-f526-42ce-b3a9-08574ba5b338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc7f9b96acb14aca890e684ef19028ba, user_id=25371fa9f9264032bc796aed558ff250, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:13.448039 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24564e7e-f526-42ce-b3a9-08574ba5b338 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:13.481235 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1040/2060] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:13 2022] PUT /identity/v3/projects/dc7f9b96acb14aca890e684ef19028ba/users/25371fa9f9264032bc796aed558ff250/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:13.483688 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17eb1f-658d-4b53-930e-95f93c83dbbf None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:13.484802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17eb1f-658d-4b53-930e-95f93c83dbbf None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:13.485434 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17eb1f-658d-4b53-930e-95f93c83dbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:13.553683 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-cd17eb1f-658d-4b53-930e-95f93c83dbbf None None] MFA Rules not processed for user `136d02137cb8458ab66659512fd11f2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:13.556251 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-cd17eb1f-658d-4b53-930e-95f93c83dbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:13.688844 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2c65d-2914-4c21-8c11-531acfd49998 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:13.741693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1041/2061] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:13.743303 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17eb1f-658d-4b53-930e-95f93c83dbbf None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:13.760782 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:13.888914 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe5e5fe-33d7-48bf-ab09-1a430e054356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:13.890738 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5e5fe-33d7-48bf-ab09-1a430e054356 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:13.891153 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5e5fe-33d7-48bf-ab09-1a430e054356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:13.891626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5e5fe-33d7-48bf-ab09-1a430e054356 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:13.907056 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe5e5fe-33d7-48bf-ab09-1a430e054356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:13.912197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe5e5fe-33d7-48bf-ab09-1a430e054356 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:13.937942 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1021/2062] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 515 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:13.939485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe5e5fe-33d7-48bf-ab09-1a430e054356 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:13.942893 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6fe5e5fe-33d7-48bf-ab09-1a430e054356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:13.962492 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17eb1f-658d-4b53-930e-95f93c83dbbf None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:13.976685 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:13.986532 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c5fd5-4739-4781-9918-922fb9cd5dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:13.989314 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c5fd5-4739-4781-9918-922fb9cd5dab admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:13.989949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c5fd5-4739-4781-9918-922fb9cd5dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:13.990612 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c5fd5-4739-4781-9918-922fb9cd5dab admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:13.995188 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c5fd5-4739-4781-9918-922fb9cd5dab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:14.007170 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c5fd5-4739-4781-9918-922fb9cd5dab admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:14.050609 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1042/2063] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:14.051780 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c5fd5-4739-4781-9918-922fb9cd5dab admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:14.061248 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe5e5fe-33d7-48bf-ab09-1a430e054356 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:14.071811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff0946-6af9-4e53-8745-c2ba9e210298 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:14.072762 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff0946-6af9-4e53-8745-c2ba9e210298 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:14.073332 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff0946-6af9-4e53-8745-c2ba9e210298 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:14.074073 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff0946-6af9-4e53-8745-c2ba9e210298 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:14.080006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c5fd5-4739-4781-9918-922fb9cd5dab admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:14.091138 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e6d783-78fb-4026-8f11-dd960b3b19c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:14.092851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6d783-78fb-4026-8f11-dd960b3b19c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:14.093174 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6d783-78fb-4026-8f11-dd960b3b19c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:14.093498 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6d783-78fb-4026-8f11-dd960b3b19c3 admin admin] PATH_INFO: `/v3/projects/0624da7c1b884dcdbe273dd39463beb2/users/5d2a6624174144f08c0cf29db713fbb8/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:14.096867 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff0946-6af9-4e53-8745-c2ba9e210298 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:14.101159 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff0946-6af9-4e53-8745-c2ba9e210298 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:14.120031 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1022/2064] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:14.122445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff0946-6af9-4e53-8745-c2ba9e210298 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:14.143135 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff0946-6af9-4e53-8745-c2ba9e210298 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:14.158557 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e6d783-78fb-4026-8f11-dd960b3b19c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0624da7c1b884dcdbe273dd39463beb2, user_id=5d2a6624174144f08c0cf29db713fbb8, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:14.161462 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31a6d9-cedc-4311-b675-60971b344836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:14.162623 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31a6d9-cedc-4311-b675-60971b344836 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:14.166801 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31a6d9-cedc-4311-b675-60971b344836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:14.168942 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e6d783-78fb-4026-8f11-dd960b3b19c3 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:14.169561 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31a6d9-cedc-4311-b675-60971b344836 admin admin] PATH_INFO: `/v3/projects/dc7f9b96acb14aca890e684ef19028ba/users/25371fa9f9264032bc796aed558ff250/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:14.179691 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef31a6d9-cedc-4311-b675-60971b344836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc7f9b96acb14aca890e684ef19028ba, user_id=25371fa9f9264032bc796aed558ff250, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:14.187424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef31a6d9-cedc-4311-b675-60971b344836 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:14.208924 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1043/2065] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:14 2022] PUT /identity/v3/projects/0624da7c1b884dcdbe273dd39463beb2/users/5d2a6624174144f08c0cf29db713fbb8/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:14.211072 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e6d783-78fb-4026-8f11-dd960b3b19c3 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:14.253826 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1023/2066] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:14 2022] PUT /identity/v3/projects/dc7f9b96acb14aca890e684ef19028ba/users/25371fa9f9264032bc796aed558ff250/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:14.258246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b013e-992d-4907-8564-a6ac52ebe4b1 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:14.260407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b013e-992d-4907-8564-a6ac52ebe4b1 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:14.260684 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b013e-992d-4907-8564-a6ac52ebe4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:14.360568 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-be9b013e-992d-4907-8564-a6ac52ebe4b1 None None] MFA Rules not processed for user `25371fa9f9264032bc796aed558ff250`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:14.367319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-be9b013e-992d-4907-8564-a6ac52ebe4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:14.497657 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e6d783-78fb-4026-8f11-dd960b3b19c3 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:14.678922 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1024/2067] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 424 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:14.758732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff9b29-34d0-4805-b1cc-937dc18601cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:14.759970 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff9b29-34d0-4805-b1cc-937dc18601cb admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:14.760275 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff9b29-34d0-4805-b1cc-937dc18601cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:14.761505 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff9b29-34d0-4805-b1cc-937dc18601cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:14.763254 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff9b29-34d0-4805-b1cc-937dc18601cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:14.766427 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff9b29-34d0-4805-b1cc-937dc18601cb admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:14.810294 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1044/2068] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 600 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:14.814060 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b013e-992d-4907-8564-a6ac52ebe4b1 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:14.843982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b013e-992d-4907-8564-a6ac52ebe4b1 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:14.856477 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeeff4d-67ce-4e65-8fbb-449543817247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:14.858454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeeff4d-67ce-4e65-8fbb-449543817247 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:14.859069 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeeff4d-67ce-4e65-8fbb-449543817247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:14.859579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeeff4d-67ce-4e65-8fbb-449543817247 admin admin] PATH_INFO: `/v3/projects/0624da7c1b884dcdbe273dd39463beb2/users/5d2a6624174144f08c0cf29db713fbb8/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:14.864707 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeeff4d-67ce-4e65-8fbb-449543817247 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0624da7c1b884dcdbe273dd39463beb2, user_id=5d2a6624174144f08c0cf29db713fbb8, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:14.874837 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeeff4d-67ce-4e65-8fbb-449543817247 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:14.914792 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1025/2069] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:14 2022] PUT /identity/v3/projects/0624da7c1b884dcdbe273dd39463beb2/users/5d2a6624174144f08c0cf29db713fbb8/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:14.921718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd57ed-7d19-4796-9fcf-040b704efd4d None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:14.922251 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd57ed-7d19-4796-9fcf-040b704efd4d None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:14.922793 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd57ed-7d19-4796-9fcf-040b704efd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:15.008650 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-aacd57ed-7d19-4796-9fcf-040b704efd4d None None] MFA Rules not processed for user `5d2a6624174144f08c0cf29db713fbb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:15.013148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-aacd57ed-7d19-4796-9fcf-040b704efd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:15.017277 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeeff4d-67ce-4e65-8fbb-449543817247 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:15.050186 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:15.116169 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeeff4d-67ce-4e65-8fbb-449543817247 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:15.124007 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc061c0-eafd-4773-b310-3978ede64bbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:15.125509 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc061c0-eafd-4773-b310-3978ede64bbe service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:15.126044 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc061c0-eafd-4773-b310-3978ede64bbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:15.126398 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc061c0-eafd-4773-b310-3978ede64bbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:15.225857 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:15.264270 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1045/2070] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:14 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:15.284247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc061c0-eafd-4773-b310-3978ede64bbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:15.291334 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc061c0-eafd-4773-b310-3978ede64bbe service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:15.312038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1026/2071] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:13:15.382278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd57ed-7d19-4796-9fcf-040b704efd4d None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:15.385469 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-aacd57ed-7d19-4796-9fcf-040b704efd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:15.408642 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd57ed-7d19-4796-9fcf-040b704efd4d None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:15.414992 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362093fe-ce54-4d4c-afd6-b6f23997cdb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:15.416625 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362093fe-ce54-4d4c-afd6-b6f23997cdb4 service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:15.417693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362093fe-ce54-4d4c-afd6-b6f23997cdb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:15.417693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362093fe-ce54-4d4c-afd6-b6f23997cdb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:15.442372 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362093fe-ce54-4d4c-afd6-b6f23997cdb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:15.446197 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362093fe-ce54-4d4c-afd6-b6f23997cdb4 service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:15.470105 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1046/2072] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:15 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:15.536448 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc061c0-eafd-4773-b310-3978ede64bbe service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:15.551686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc061c0-eafd-4773-b310-3978ede64bbe service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:15.558384 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5f0d1-6686-4ecd-b67b-cd61e3fa3b4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:15.559640 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5f0d1-6686-4ecd-b67b-cd61e3fa3b4b service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:15.560239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5f0d1-6686-4ecd-b67b-cd61e3fa3b4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:15.560819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5f0d1-6686-4ecd-b67b-cd61e3fa3b4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:15.722653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e5f0d1-6686-4ecd-b67b-cd61e3fa3b4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:15.725777 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e5f0d1-6686-4ecd-b67b-cd61e3fa3b4b service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:15.754346 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1027/2073] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:15 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:17.331072 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362093fe-ce54-4d4c-afd6-b6f23997cdb4 service cinder] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:17.346703 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362093fe-ce54-4d4c-afd6-b6f23997cdb4 service cinder] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:17.352230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac1eac-ba71-495c-bc5f-cf358fe8468a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:17.352995 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac1eac-ba71-495c-bc5f-cf358fe8468a service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:17.353325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac1eac-ba71-495c-bc5f-cf358fe8468a service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:17.353626 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac1eac-ba71-495c-bc5f-cf358fe8468a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:17.446848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ac1eac-ba71-495c-bc5f-cf358fe8468a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:17.449243 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ac1eac-ba71-495c-bc5f-cf358fe8468a service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:17.472639 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1047/2074] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:18.743054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5f0d1-6686-4ecd-b67b-cd61e3fa3b4b service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:18.759295 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5f0d1-6686-4ecd-b67b-cd61e3fa3b4b service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:18.766726 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35bda3-977f-4db4-853e-e507048d3640 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:18.767806 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35bda3-977f-4db4-853e-e507048d3640 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:18.768260 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35bda3-977f-4db4-853e-e507048d3640 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:18.768812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35bda3-977f-4db4-853e-e507048d3640 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:18.900042 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd35bda3-977f-4db4-853e-e507048d3640 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:18.903448 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd35bda3-977f-4db4-853e-e507048d3640 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:18.924850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1028/2075] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:18 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:20.115296 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac1eac-ba71-495c-bc5f-cf358fe8468a service cinder] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:20.225981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac1eac-ba71-495c-bc5f-cf358fe8468a service cinder] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:20.233467 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bebc1b-3422-46de-887f-ee4d7d04d03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:20.235062 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bebc1b-3422-46de-887f-ee4d7d04d03b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:20.235391 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bebc1b-3422-46de-887f-ee4d7d04d03b admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:20.235710 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bebc1b-3422-46de-887f-ee4d7d04d03b admin admin] PATH_INFO: `/v3/users/eabca44411d04808a4c8862728b226e0` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:20.237410 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bebc1b-3422-46de-887f-ee4d7d04d03b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eabca44411d04808a4c8862728b226e0)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:20.240595 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bebc1b-3422-46de-887f-ee4d7d04d03b admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:20.317812 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:20.341748 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-47bebc1b-3422-46de-887f-ee4d7d04d03b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eabca44411d04808a4c8862728b226e0'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:20.375034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-47bebc1b-3422-46de-887f-ee4d7d04d03b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eabca44411d04808a4c8862728b226e0'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:20.416758 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-47bebc1b-3422-46de-887f-ee4d7d04d03b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eabca44411d04808a4c8862728b226e0'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:20.420786 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-47bebc1b-3422-46de-887f-ee4d7d04d03b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eabca44411d04808a4c8862728b226e0'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:20.432296 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-47bebc1b-3422-46de-887f-ee4d7d04d03b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eabca44411d04808a4c8862728b226e0'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:20.447719 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1048/2076] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:20 2022] DELETE /identity/v3/users/eabca44411d04808a4c8862728b226e0 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:20.462552 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a8075-9470-4f97-89df-9f6c57f27500 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:20.463207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a8075-9470-4f97-89df-9f6c57f27500 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:20.463646 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a8075-9470-4f97-89df-9f6c57f27500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:20.487559 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-936a8075-9470-4f97-89df-9f6c57f27500 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:20.489591 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-936a8075-9470-4f97-89df-9f6c57f27500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:20.585063 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1029/2077] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:20.596903 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bebc1b-3422-46de-887f-ee4d7d04d03b admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:20.598937 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-47bebc1b-3422-46de-887f-ee4d7d04d03b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:20.702535 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bebc1b-3422-46de-887f-ee4d7d04d03b admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:20.709677 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014f4d8-cde6-4419-9799-f1b3cbd78343 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:20.710949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014f4d8-cde6-4419-9799-f1b3cbd78343 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:20.711706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014f4d8-cde6-4419-9799-f1b3cbd78343 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:20.712223 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014f4d8-cde6-4419-9799-f1b3cbd78343 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:20.726088 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3014f4d8-cde6-4419-9799-f1b3cbd78343 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:20.727841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3014f4d8-cde6-4419-9799-f1b3cbd78343 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:20.745367 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1049/2078] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:21.572080 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a8075-9470-4f97-89df-9f6c57f27500 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:21.575475 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-936a8075-9470-4f97-89df-9f6c57f27500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:21.600472 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:21.618322 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a8075-9470-4f97-89df-9f6c57f27500 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:21.631716 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac3853-b3f0-4bce-9198-3c93daff7471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:21.632869 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac3853-b3f0-4bce-9198-3c93daff7471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:21.633209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac3853-b3f0-4bce-9198-3c93daff7471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:21.633606 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac3853-b3f0-4bce-9198-3c93daff7471 admin admin] PATH_INFO: `/v3/projects/7b72cdf8559f4a949fa568b877ea1080` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:21.645528 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ac3853-b3f0-4bce-9198-3c93daff7471 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b72cdf8559f4a949fa568b877ea1080)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:21.654298 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ac3853-b3f0-4bce-9198-3c93daff7471 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:21.822180 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-73ac3853-b3f0-4bce-9198-3c93daff7471 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b72cdf8559f4a949fa568b877ea1080', 'request_id': 'req-73ac3853-b3f0-4bce-9198-3c93daff7471'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:21.843644 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-73ac3853-b3f0-4bce-9198-3c93daff7471 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b72cdf8559f4a949fa568b877ea1080', 'request_id': 'req-73ac3853-b3f0-4bce-9198-3c93daff7471'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:21.860851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-73ac3853-b3f0-4bce-9198-3c93daff7471 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b72cdf8559f4a949fa568b877ea1080', 'request_id': 'req-73ac3853-b3f0-4bce-9198-3c93daff7471'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:21.891744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-73ac3853-b3f0-4bce-9198-3c93daff7471 admin admin] The token cache is being invalidate because project 7b72cdf8559f4a949fa568b877ea1080 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:21.892018 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-73ac3853-b3f0-4bce-9198-3c93daff7471 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:21.896461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-73ac3853-b3f0-4bce-9198-3c93daff7471 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:21.906279 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1030/2079] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:21 2022] DELETE /identity/v3/projects/7b72cdf8559f4a949fa568b877ea1080 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:21.907498 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014f4d8-cde6-4419-9799-f1b3cbd78343 service neutron] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:21.931003 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014f4d8-cde6-4419-9799-f1b3cbd78343 service neutron] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:22.153051 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797dc63-6135-4e50-8fb6-6c071bbe01e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:22.155085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797dc63-6135-4e50-8fb6-6c071bbe01e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:22.155985 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797dc63-6135-4e50-8fb6-6c071bbe01e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:22.156856 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797dc63-6135-4e50-8fb6-6c071bbe01e6 admin admin] PATH_INFO: `/v3/users/5d2a6624174144f08c0cf29db713fbb8` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:22.160242 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2797dc63-6135-4e50-8fb6-6c071bbe01e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d2a6624174144f08c0cf29db713fbb8)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:22.163167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2797dc63-6135-4e50-8fb6-6c071bbe01e6 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:22.295703 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-2797dc63-6135-4e50-8fb6-6c071bbe01e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d2a6624174144f08c0cf29db713fbb8'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:22.307755 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:22.320340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-2797dc63-6135-4e50-8fb6-6c071bbe01e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d2a6624174144f08c0cf29db713fbb8'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:22.337274 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-2797dc63-6135-4e50-8fb6-6c071bbe01e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d2a6624174144f08c0cf29db713fbb8'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:22.339219 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-2797dc63-6135-4e50-8fb6-6c071bbe01e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d2a6624174144f08c0cf29db713fbb8'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:22.353717 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-2797dc63-6135-4e50-8fb6-6c071bbe01e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d2a6624174144f08c0cf29db713fbb8'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:22.372751 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1050/2080] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:21 2022] DELETE /identity/v3/users/5d2a6624174144f08c0cf29db713fbb8 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:22.850598 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac3853-b3f0-4bce-9198-3c93daff7471 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:22.852445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-73ac3853-b3f0-4bce-9198-3c93daff7471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:22.895357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac3853-b3f0-4bce-9198-3c93daff7471 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:22.964596 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3ff83-6b77-4993-9862-3dad2908e008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:22.965512 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3ff83-6b77-4993-9862-3dad2908e008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:22.965808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3ff83-6b77-4993-9862-3dad2908e008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:22.966161 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3ff83-6b77-4993-9862-3dad2908e008 admin admin] PATH_INFO: `/v3/projects/0624da7c1b884dcdbe273dd39463beb2` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:22.967449 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3ff83-6b77-4993-9862-3dad2908e008 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0624da7c1b884dcdbe273dd39463beb2)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:22.969479 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3ff83-6b77-4993-9862-3dad2908e008 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:23.054901 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2db3ff83-6b77-4993-9862-3dad2908e008 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0624da7c1b884dcdbe273dd39463beb2', 'request_id': 'req-2db3ff83-6b77-4993-9862-3dad2908e008'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:23.063001 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2db3ff83-6b77-4993-9862-3dad2908e008 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0624da7c1b884dcdbe273dd39463beb2', 'request_id': 'req-2db3ff83-6b77-4993-9862-3dad2908e008'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:23.075256 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2db3ff83-6b77-4993-9862-3dad2908e008 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0624da7c1b884dcdbe273dd39463beb2', 'request_id': 'req-2db3ff83-6b77-4993-9862-3dad2908e008'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:23.093703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2db3ff83-6b77-4993-9862-3dad2908e008 admin admin] The token cache is being invalidate because project 0624da7c1b884dcdbe273dd39463beb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:23.094073 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2db3ff83-6b77-4993-9862-3dad2908e008 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:23.094698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2db3ff83-6b77-4993-9862-3dad2908e008 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:23.097075 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1031/2081] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:22 2022] DELETE /identity/v3/projects/0624da7c1b884dcdbe273dd39463beb2 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:23.104222 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931e995-9024-4b0f-98a9-40ca5d649052 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:23.106907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931e995-9024-4b0f-98a9-40ca5d649052 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:23.106907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931e995-9024-4b0f-98a9-40ca5d649052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:23.146597 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-c931e995-9024-4b0f-98a9-40ca5d649052 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:23.148059 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-c931e995-9024-4b0f-98a9-40ca5d649052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:23.241220 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1051/2082] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:23.249160 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16168b8-4f7f-4dcb-893c-30782d45217f None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:23.249564 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16168b8-4f7f-4dcb-893c-30782d45217f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:23.249807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16168b8-4f7f-4dcb-893c-30782d45217f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:23.278428 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-a16168b8-4f7f-4dcb-893c-30782d45217f None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:23.280519 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a16168b8-4f7f-4dcb-893c-30782d45217f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:23.294108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1032/2083] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:13:23.300244 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931e995-9024-4b0f-98a9-40ca5d649052 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:23.310688 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931e995-9024-4b0f-98a9-40ca5d649052 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:23.315765 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82465159-3cd2-4e23-968a-a3e9b6460d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:23.316439 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82465159-3cd2-4e23-968a-a3e9b6460d41 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:23.316716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82465159-3cd2-4e23-968a-a3e9b6460d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:23.316992 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82465159-3cd2-4e23-968a-a3e9b6460d41 admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:23.317724 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82465159-3cd2-4e23-968a-a3e9b6460d41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:13:23.318063 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82465159-3cd2-4e23-968a-a3e9b6460d41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:23.319680 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82465159-3cd2-4e23-968a-a3e9b6460d41 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:23.335960 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1052/2084] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:13:23.343396 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16168b8-4f7f-4dcb-893c-30782d45217f None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:23.358613 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16168b8-4f7f-4dcb-893c-30782d45217f None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:23.364020 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58c24a-4e20-456a-b8bb-07495842fdd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:23.364691 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58c24a-4e20-456a-b8bb-07495842fdd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:23.364978 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58c24a-4e20-456a-b8bb-07495842fdd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:23.365256 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58c24a-4e20-456a-b8bb-07495842fdd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:23.365973 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff58c24a-4e20-456a-b8bb-07495842fdd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:23.367898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff58c24a-4e20-456a-b8bb-07495842fdd2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:23.372482 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:23.408733 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1033/2085] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:23 2022] POST /identity/v3/projects => generated 377 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:13:23.412851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82465159-3cd2-4e23-968a-a3e9b6460d41 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:23.423529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82465159-3cd2-4e23-968a-a3e9b6460d41 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:23.515412 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab441a3-a6ed-4135-b8d9-9816a100f0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:23.516390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab441a3-a6ed-4135-b8d9-9816a100f0cf admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:23.516865 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab441a3-a6ed-4135-b8d9-9816a100f0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:23.517275 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab441a3-a6ed-4135-b8d9-9816a100f0cf admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:23.518216 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab441a3-a6ed-4135-b8d9-9816a100f0cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:23.521270 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab441a3-a6ed-4135-b8d9-9816a100f0cf admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:23.560169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1053/2086] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:23 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:23.565973 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58c24a-4e20-456a-b8bb-07495842fdd2 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:23.580888 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58c24a-4e20-456a-b8bb-07495842fdd2 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:23.587943 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5488f2c-a0ca-49da-9ea2-b57ffa10530b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:23.590077 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5488f2c-a0ca-49da-9ea2-b57ffa10530b admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:23.590491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5488f2c-a0ca-49da-9ea2-b57ffa10530b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:23.591055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5488f2c-a0ca-49da-9ea2-b57ffa10530b admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:23.592394 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5488f2c-a0ca-49da-9ea2-b57ffa10530b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:23.594630 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5488f2c-a0ca-49da-9ea2-b57ffa10530b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:23.610194 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1034/2087] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:23.618502 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab441a3-a6ed-4135-b8d9-9816a100f0cf admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:23.631266 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab441a3-a6ed-4135-b8d9-9816a100f0cf admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:23.640133 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e071b-3e00-4120-a2dd-4644bb2f080b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:23.640815 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e071b-3e00-4120-a2dd-4644bb2f080b admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:23.641102 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e071b-3e00-4120-a2dd-4644bb2f080b admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:23.641377 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e071b-3e00-4120-a2dd-4644bb2f080b admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:23.642154 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e071b-3e00-4120-a2dd-4644bb2f080b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:23.643767 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e071b-3e00-4120-a2dd-4644bb2f080b admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:23.656797 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1054/2088] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:23.661631 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5488f2c-a0ca-49da-9ea2-b57ffa10530b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:23.673356 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5488f2c-a0ca-49da-9ea2-b57ffa10530b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:23.679159 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af444543-acef-4c8b-b6b1-ada231342f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:23.680053 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af444543-acef-4c8b-b6b1-ada231342f67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:23.680336 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af444543-acef-4c8b-b6b1-ada231342f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:23.680617 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af444543-acef-4c8b-b6b1-ada231342f67 admin admin] PATH_INFO: `/v3/projects/591ca27a4da1433d8a620709b6025ffe/users/6122dacf4e9f4415b104fb7f7f2f1585/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:23.704967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af444543-acef-4c8b-b6b1-ada231342f67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=591ca27a4da1433d8a620709b6025ffe, user_id=6122dacf4e9f4415b104fb7f7f2f1585, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:23.708398 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af444543-acef-4c8b-b6b1-ada231342f67 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:23.733348 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1035/2089] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:23 2022] PUT /identity/v3/projects/591ca27a4da1433d8a620709b6025ffe/users/6122dacf4e9f4415b104fb7f7f2f1585/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:23.749062 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905efe9c-5927-4b17-bd3e-371678676ac5 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:23.749542 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905efe9c-5927-4b17-bd3e-371678676ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:23.750036 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905efe9c-5927-4b17-bd3e-371678676ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:23.788666 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-905efe9c-5927-4b17-bd3e-371678676ac5 None None] MFA Rules not processed for user `6122dacf4e9f4415b104fb7f7f2f1585`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:23.790267 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-905efe9c-5927-4b17-bd3e-371678676ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:23.825834 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:23.892185 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1055/2090] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:23.898845 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af444543-acef-4c8b-b6b1-ada231342f67 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:23.992575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af444543-acef-4c8b-b6b1-ada231342f67 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:24.000538 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc657f-f264-4de4-b63c-a5e91e14cb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:24.001383 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc657f-f264-4de4-b63c-a5e91e14cb4d admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:24.001745 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc657f-f264-4de4-b63c-a5e91e14cb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:24.002254 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc657f-f264-4de4-b63c-a5e91e14cb4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:24.003464 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc657f-f264-4de4-b63c-a5e91e14cb4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:24.006453 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc657f-f264-4de4-b63c-a5e91e14cb4d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:24.066533 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1036/2091] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:23 2022] POST /identity/v3/projects => generated 375 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:13:24.072141 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905efe9c-5927-4b17-bd3e-371678676ac5 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:24.088729 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905efe9c-5927-4b17-bd3e-371678676ac5 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:24.182907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e703fe-ebf8-4ba3-a87d-05cc8727ba9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:24.183950 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e703fe-ebf8-4ba3-a87d-05cc8727ba9e admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:24.184146 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e703fe-ebf8-4ba3-a87d-05cc8727ba9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:24.184429 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e703fe-ebf8-4ba3-a87d-05cc8727ba9e admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:24.185214 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e703fe-ebf8-4ba3-a87d-05cc8727ba9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:24.187278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e703fe-ebf8-4ba3-a87d-05cc8727ba9e admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:24.223554 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1056/2092] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:24 2022] POST /identity/v3/users => generated 288 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:24.228164 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc657f-f264-4de4-b63c-a5e91e14cb4d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:24.243851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc657f-f264-4de4-b63c-a5e91e14cb4d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:24.250265 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b3c8d-1053-4baf-a88f-3b1d308e310b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:24.251258 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b3c8d-1053-4baf-a88f-3b1d308e310b admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:24.251642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b3c8d-1053-4baf-a88f-3b1d308e310b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:24.252061 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b3c8d-1053-4baf-a88f-3b1d308e310b admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:24.253281 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090b3c8d-1053-4baf-a88f-3b1d308e310b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:24.255963 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090b3c8d-1053-4baf-a88f-3b1d308e310b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:24.273149 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1037/2093] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:24.280267 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e703fe-ebf8-4ba3-a87d-05cc8727ba9e admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:24.296819 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e703fe-ebf8-4ba3-a87d-05cc8727ba9e admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:24.306385 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db14d2a-af09-4f0b-8cfd-df10fd5475bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:24.308271 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db14d2a-af09-4f0b-8cfd-df10fd5475bd admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:24.308811 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db14d2a-af09-4f0b-8cfd-df10fd5475bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:24.309299 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db14d2a-af09-4f0b-8cfd-df10fd5475bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:24.310156 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db14d2a-af09-4f0b-8cfd-df10fd5475bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:24.311993 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db14d2a-af09-4f0b-8cfd-df10fd5475bd admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:24.337545 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1057/2094] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:13:24.341175 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b3c8d-1053-4baf-a88f-3b1d308e310b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:24.365865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b3c8d-1053-4baf-a88f-3b1d308e310b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:24.374891 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4d29e-284c-45a8-8abd-3ff716a7bbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:24.376975 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4d29e-284c-45a8-8abd-3ff716a7bbc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:24.376975 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4d29e-284c-45a8-8abd-3ff716a7bbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:24.377347 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4d29e-284c-45a8-8abd-3ff716a7bbc5 admin admin] PATH_INFO: `/v3/projects/80d4a4c1f4ad475b95f54ba5971736ae/users/7a63d66ffe664428b381a708d11e4a3f/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:24.414595 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd4d29e-284c-45a8-8abd-3ff716a7bbc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80d4a4c1f4ad475b95f54ba5971736ae, user_id=7a63d66ffe664428b381a708d11e4a3f, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:24.425394 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd4d29e-284c-45a8-8abd-3ff716a7bbc5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:24.467426 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1038/2095] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:24 2022] PUT /identity/v3/projects/80d4a4c1f4ad475b95f54ba5971736ae/users/7a63d66ffe664428b381a708d11e4a3f/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:24.480821 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48782c75-fc02-40d8-a1c1-58032c06991c None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:24.481115 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48782c75-fc02-40d8-a1c1-58032c06991c None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:24.481397 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48782c75-fc02-40d8-a1c1-58032c06991c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:24.528696 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-48782c75-fc02-40d8-a1c1-58032c06991c None None] MFA Rules not processed for user `7a63d66ffe664428b381a708d11e4a3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:24.531133 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-48782c75-fc02-40d8-a1c1-58032c06991c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:24.639356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1058/2096] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:24.644728 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4d29e-284c-45a8-8abd-3ff716a7bbc5 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:24.740137 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4d29e-284c-45a8-8abd-3ff716a7bbc5 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:24.747416 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be70e514-fb60-49f2-b82c-efd60546f253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:24.748115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70e514-fb60-49f2-b82c-efd60546f253 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:24.748495 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70e514-fb60-49f2-b82c-efd60546f253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:24.748681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70e514-fb60-49f2-b82c-efd60546f253 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:24.749391 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be70e514-fb60-49f2-b82c-efd60546f253 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:24.753086 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be70e514-fb60-49f2-b82c-efd60546f253 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:24.759210 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:24.801850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1039/2097] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:24 2022] POST /identity/v3/projects => generated 377 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:24.806122 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48782c75-fc02-40d8-a1c1-58032c06991c None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:24.820018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48782c75-fc02-40d8-a1c1-58032c06991c None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:24.871655 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:24.905443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae4d51-4961-4031-a7e9-8b7881f2e1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:24.906193 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae4d51-4961-4031-a7e9-8b7881f2e1a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:24.906484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae4d51-4961-4031-a7e9-8b7881f2e1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:24.911048 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae4d51-4961-4031-a7e9-8b7881f2e1a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:24.911048 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae4d51-4961-4031-a7e9-8b7881f2e1a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:24.911048 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae4d51-4961-4031-a7e9-8b7881f2e1a1 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:24.940036 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1059/2098] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:24 2022] POST /identity/v3/users => generated 295 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:24.946650 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be70e514-fb60-49f2-b82c-efd60546f253 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:24.966017 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be70e514-fb60-49f2-b82c-efd60546f253 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:24.971198 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f5638-62f8-45c2-a08a-b34042aabbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:24.971898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f5638-62f8-45c2-a08a-b34042aabbfd admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:24.972183 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f5638-62f8-45c2-a08a-b34042aabbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:24.972458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f5638-62f8-45c2-a08a-b34042aabbfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:24.973197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92f5638-62f8-45c2-a08a-b34042aabbfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:24.974883 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92f5638-62f8-45c2-a08a-b34042aabbfd admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:24.987580 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1040/2099] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:24.992737 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae4d51-4961-4031-a7e9-8b7881f2e1a1 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:25.012170 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae4d51-4961-4031-a7e9-8b7881f2e1a1 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:25.023940 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6566b-ef90-403b-8367-80f1342302e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:25.025515 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6566b-ef90-403b-8367-80f1342302e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:25.026178 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6566b-ef90-403b-8367-80f1342302e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:25.027350 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6566b-ef90-403b-8367-80f1342302e1 admin admin] PATH_INFO: `/v3/projects/f13bd6240aa34f0e84612c1a77e4715b/users/1ca8abb561864565a4c70a2fb8eaa0d6/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:25.047153 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f6566b-ef90-403b-8367-80f1342302e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f13bd6240aa34f0e84612c1a77e4715b, user_id=1ca8abb561864565a4c70a2fb8eaa0d6, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:25.049886 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f6566b-ef90-403b-8367-80f1342302e1 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:25.066598 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1060/2100] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:24 2022] PUT /identity/v3/projects/f13bd6240aa34f0e84612c1a77e4715b/users/1ca8abb561864565a4c70a2fb8eaa0d6/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:25.070947 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f5638-62f8-45c2-a08a-b34042aabbfd admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:25.084927 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f5638-62f8-45c2-a08a-b34042aabbfd admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:25.187470 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839cc1c5-5f54-4b31-87f3-f3ffc497e2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:25.188414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cc1c5-5f54-4b31-87f3-f3ffc497e2e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:25.188726 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cc1c5-5f54-4b31-87f3-f3ffc497e2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:25.189041 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cc1c5-5f54-4b31-87f3-f3ffc497e2e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:25.189907 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839cc1c5-5f54-4b31-87f3-f3ffc497e2e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:25.191808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839cc1c5-5f54-4b31-87f3-f3ffc497e2e0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:25.204249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1041/2101] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:25.208776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6566b-ef90-403b-8367-80f1342302e1 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:25.221835 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6566b-ef90-403b-8367-80f1342302e1 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:25.228744 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe6278-d03b-4a8e-9d10-2044eceee56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:25.230125 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe6278-d03b-4a8e-9d10-2044eceee56d admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:25.230603 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe6278-d03b-4a8e-9d10-2044eceee56d admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:25.230965 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe6278-d03b-4a8e-9d10-2044eceee56d admin admin] PATH_INFO: `/v3/projects/f13bd6240aa34f0e84612c1a77e4715b/users/1ca8abb561864565a4c70a2fb8eaa0d6/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:25.233842 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe6278-d03b-4a8e-9d10-2044eceee56d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f13bd6240aa34f0e84612c1a77e4715b, user_id=1ca8abb561864565a4c70a2fb8eaa0d6, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:25.237281 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe6278-d03b-4a8e-9d10-2044eceee56d admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:25.253308 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1061/2102] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:25 2022] PUT /identity/v3/projects/f13bd6240aa34f0e84612c1a77e4715b/users/1ca8abb561864565a4c70a2fb8eaa0d6/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:25.270942 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d34e95-628f-46d9-a554-b5ecfb27c349 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:25.272041 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d34e95-628f-46d9-a554-b5ecfb27c349 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:25.272041 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d34e95-628f-46d9-a554-b5ecfb27c349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:25.327919 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-09d34e95-628f-46d9-a554-b5ecfb27c349 None None] MFA Rules not processed for user `1ca8abb561864565a4c70a2fb8eaa0d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:25.329920 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-09d34e95-628f-46d9-a554-b5ecfb27c349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:25.420538 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1042/2103] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:25 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:13:25.433670 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe6278-d03b-4a8e-9d10-2044eceee56d admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:25.436208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-3ffe6278-d03b-4a8e-9d10-2044eceee56d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:25.553149 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe6278-d03b-4a8e-9d10-2044eceee56d admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:25.559658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4490582-f13b-4bf8-9e84-6c7485bd6426 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:25.560619 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4490582-f13b-4bf8-9e84-6c7485bd6426 service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:25.560909 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4490582-f13b-4bf8-9e84-6c7485bd6426 service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:25.561167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4490582-f13b-4bf8-9e84-6c7485bd6426 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:25.672692 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4490582-f13b-4bf8-9e84-6c7485bd6426 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:25.676476 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4490582-f13b-4bf8-9e84-6c7485bd6426 service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:25.698842 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1062/2104] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:27.571528 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d34e95-628f-46d9-a554-b5ecfb27c349 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:27.588229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d34e95-628f-46d9-a554-b5ecfb27c349 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:27.593093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857636c-246b-41d7-9a77-03fdd357d3a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:27.594289 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857636c-246b-41d7-9a77-03fdd357d3a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:27.594718 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857636c-246b-41d7-9a77-03fdd357d3a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:27.595137 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857636c-246b-41d7-9a77-03fdd357d3a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:27.687496 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:27.696726 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b857636c-246b-41d7-9a77-03fdd357d3a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:27.702147 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b857636c-246b-41d7-9a77-03fdd357d3a7 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:27.736297 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1043/2105] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:31.330358 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4490582-f13b-4bf8-9e84-6c7485bd6426 service cinder] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:31.343047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4490582-f13b-4bf8-9e84-6c7485bd6426 service cinder] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:31.348629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef066c-5c72-4c13-83bd-949b1750cfc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:31.349293 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef066c-5c72-4c13-83bd-949b1750cfc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:31.349572 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef066c-5c72-4c13-83bd-949b1750cfc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:31.349981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef066c-5c72-4c13-83bd-949b1750cfc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:31.360536 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ef066c-5c72-4c13-83bd-949b1750cfc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:31.362497 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ef066c-5c72-4c13-83bd-949b1750cfc4 service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:31.378120 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1063/2106] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:31 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:32.501714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857636c-246b-41d7-9a77-03fdd357d3a7 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:32.505533 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b857636c-246b-41d7-9a77-03fdd357d3a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:32.647974 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857636c-246b-41d7-9a77-03fdd357d3a7 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:32.656188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4381a-f147-45ca-97c4-d29a3b3215b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:32.658436 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4381a-f147-45ca-97c4-d29a3b3215b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:32.658988 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4381a-f147-45ca-97c4-d29a3b3215b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:32.659447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4381a-f147-45ca-97c4-d29a3b3215b5 admin admin] PATH_INFO: `/v3/users/0ea596fe816948438df6d9648348e6e9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:32.662157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c4381a-f147-45ca-97c4-d29a3b3215b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ea596fe816948438df6d9648348e6e9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:32.665781 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c4381a-f147-45ca-97c4-d29a3b3215b5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:32.828657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-18c4381a-f147-45ca-97c4-d29a3b3215b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ea596fe816948438df6d9648348e6e9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:32.857818 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-18c4381a-f147-45ca-97c4-d29a3b3215b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ea596fe816948438df6d9648348e6e9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:32.877843 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-18c4381a-f147-45ca-97c4-d29a3b3215b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ea596fe816948438df6d9648348e6e9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:32.889897 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-18c4381a-f147-45ca-97c4-d29a3b3215b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ea596fe816948438df6d9648348e6e9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:32.890936 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-18c4381a-f147-45ca-97c4-d29a3b3215b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ea596fe816948438df6d9648348e6e9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:32.899766 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1044/2107] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:32 2022] DELETE /identity/v3/users/0ea596fe816948438df6d9648348e6e9 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:32.904415 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28935a48-ff8f-4b1e-879c-764af3773da5 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:32.905232 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28935a48-ff8f-4b1e-879c-764af3773da5 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:32.905232 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28935a48-ff8f-4b1e-879c-764af3773da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:32.942092 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-28935a48-ff8f-4b1e-879c-764af3773da5 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:32.945263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-28935a48-ff8f-4b1e-879c-764af3773da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:33.000105 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:33.081097 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1064/2108] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:33.100086 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4381a-f147-45ca-97c4-d29a3b3215b5 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:33.105436 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-18c4381a-f147-45ca-97c4-d29a3b3215b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:33.343686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4381a-f147-45ca-97c4-d29a3b3215b5 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:33.349299 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cd48b-d485-41c2-bf4c-61d65616740d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:33.350019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cd48b-d485-41c2-bf4c-61d65616740d service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:33.350310 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cd48b-d485-41c2-bf4c-61d65616740d service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:33.350587 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cd48b-d485-41c2-bf4c-61d65616740d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:33.366182 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52cd48b-d485-41c2-bf4c-61d65616740d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:33.368673 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52cd48b-d485-41c2-bf4c-61d65616740d service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:33.387384 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1045/2109] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:33.848162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28935a48-ff8f-4b1e-879c-764af3773da5 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:33.851660 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-28935a48-ff8f-4b1e-879c-764af3773da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:33.887982 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28935a48-ff8f-4b1e-879c-764af3773da5 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:33.895400 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aae805-2e60-4c57-9a8e-743c4b95cae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:33.896672 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aae805-2e60-4c57-9a8e-743c4b95cae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:33.897062 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aae805-2e60-4c57-9a8e-743c4b95cae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:33.897481 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aae805-2e60-4c57-9a8e-743c4b95cae8 admin admin] PATH_INFO: `/v3/projects/e53005358da24611a3e97ae2b5073b99` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:33.899391 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aae805-2e60-4c57-9a8e-743c4b95cae8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e53005358da24611a3e97ae2b5073b99)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:33.904015 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aae805-2e60-4c57-9a8e-743c4b95cae8 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:33.967724 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-c4aae805-2e60-4c57-9a8e-743c4b95cae8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e53005358da24611a3e97ae2b5073b99', 'request_id': 'req-c4aae805-2e60-4c57-9a8e-743c4b95cae8'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:33.974514 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-c4aae805-2e60-4c57-9a8e-743c4b95cae8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e53005358da24611a3e97ae2b5073b99', 'request_id': 'req-c4aae805-2e60-4c57-9a8e-743c4b95cae8'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:33.985987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-c4aae805-2e60-4c57-9a8e-743c4b95cae8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e53005358da24611a3e97ae2b5073b99', 'request_id': 'req-c4aae805-2e60-4c57-9a8e-743c4b95cae8'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:34.004078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-c4aae805-2e60-4c57-9a8e-743c4b95cae8 admin admin] The token cache is being invalidate because project e53005358da24611a3e97ae2b5073b99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:34.004494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-c4aae805-2e60-4c57-9a8e-743c4b95cae8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:34.005315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-c4aae805-2e60-4c57-9a8e-743c4b95cae8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:34.008127 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1065/2110] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:33 2022] DELETE /identity/v3/projects/e53005358da24611a3e97ae2b5073b99 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:34.013019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cd48b-d485-41c2-bf4c-61d65616740d service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:34.023543 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cd48b-d485-41c2-bf4c-61d65616740d service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:34.045064 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:34.118058 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66acae3-751d-46d8-abdc-5c7611a1e368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:34.119026 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66acae3-751d-46d8-abdc-5c7611a1e368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:34.119405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66acae3-751d-46d8-abdc-5c7611a1e368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:34.119778 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66acae3-751d-46d8-abdc-5c7611a1e368 admin admin] PATH_INFO: `/v3/users/1096eecae1fb4d318da932e2b440a4f4` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:34.121329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66acae3-751d-46d8-abdc-5c7611a1e368 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1096eecae1fb4d318da932e2b440a4f4)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:34.123988 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66acae3-751d-46d8-abdc-5c7611a1e368 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:34.190928 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e66acae3-751d-46d8-abdc-5c7611a1e368 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1096eecae1fb4d318da932e2b440a4f4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:34.210623 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e66acae3-751d-46d8-abdc-5c7611a1e368 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1096eecae1fb4d318da932e2b440a4f4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:34.223410 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e66acae3-751d-46d8-abdc-5c7611a1e368 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1096eecae1fb4d318da932e2b440a4f4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:34.254523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e66acae3-751d-46d8-abdc-5c7611a1e368 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1096eecae1fb4d318da932e2b440a4f4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:34.255390 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e66acae3-751d-46d8-abdc-5c7611a1e368 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1096eecae1fb4d318da932e2b440a4f4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:34.268468 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1046/2111] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:34 2022] DELETE /identity/v3/users/1096eecae1fb4d318da932e2b440a4f4 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:35.222969 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aae805-2e60-4c57-9a8e-743c4b95cae8 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:35.225256 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-c4aae805-2e60-4c57-9a8e-743c4b95cae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:35.253496 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aae805-2e60-4c57-9a8e-743c4b95cae8 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:35.344351 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87bfb05-dce1-4c04-a87f-99ede906dfe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:35.345388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bfb05-dce1-4c04-a87f-99ede906dfe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:35.345758 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bfb05-dce1-4c04-a87f-99ede906dfe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:35.346137 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bfb05-dce1-4c04-a87f-99ede906dfe2 admin admin] PATH_INFO: `/v3/projects/b6fb159380c44bba94ad1745856c1697` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:35.347651 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87bfb05-dce1-4c04-a87f-99ede906dfe2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6fb159380c44bba94ad1745856c1697)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:35.349971 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87bfb05-dce1-4c04-a87f-99ede906dfe2 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:35.443236 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-a87bfb05-dce1-4c04-a87f-99ede906dfe2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6fb159380c44bba94ad1745856c1697', 'request_id': 'req-a87bfb05-dce1-4c04-a87f-99ede906dfe2'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:35.449388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:35.454773 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-a87bfb05-dce1-4c04-a87f-99ede906dfe2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6fb159380c44bba94ad1745856c1697', 'request_id': 'req-a87bfb05-dce1-4c04-a87f-99ede906dfe2'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:35.469291 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-a87bfb05-dce1-4c04-a87f-99ede906dfe2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6fb159380c44bba94ad1745856c1697', 'request_id': 'req-a87bfb05-dce1-4c04-a87f-99ede906dfe2'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:35.491360 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-a87bfb05-dce1-4c04-a87f-99ede906dfe2 admin admin] The token cache is being invalidate because project b6fb159380c44bba94ad1745856c1697 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:35.491709 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-a87bfb05-dce1-4c04-a87f-99ede906dfe2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:35.492596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-a87bfb05-dce1-4c04-a87f-99ede906dfe2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:35.499808 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1066/2112] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:35 2022] DELETE /identity/v3/projects/b6fb159380c44bba94ad1745856c1697 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:35.505369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66acae3-751d-46d8-abdc-5c7611a1e368 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:35.508036 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e66acae3-751d-46d8-abdc-5c7611a1e368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:35.524776 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66acae3-751d-46d8-abdc-5c7611a1e368 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:35.630364 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a0b59-84a4-43d1-ba44-c99d781b6896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:35.631255 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a0b59-84a4-43d1-ba44-c99d781b6896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:35.631540 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a0b59-84a4-43d1-ba44-c99d781b6896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:35.631818 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a0b59-84a4-43d1-ba44-c99d781b6896 admin admin] PATH_INFO: `/v3/users/25371fa9f9264032bc796aed558ff250` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:35.634107 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a0b59-84a4-43d1-ba44-c99d781b6896 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25371fa9f9264032bc796aed558ff250)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:35.637995 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a0b59-84a4-43d1-ba44-c99d781b6896 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:35.718684 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:35.731208 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c18a0b59-84a4-43d1-ba44-c99d781b6896 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25371fa9f9264032bc796aed558ff250'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:35.758206 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c18a0b59-84a4-43d1-ba44-c99d781b6896 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25371fa9f9264032bc796aed558ff250'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:35.772567 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c18a0b59-84a4-43d1-ba44-c99d781b6896 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25371fa9f9264032bc796aed558ff250'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:35.789080 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c18a0b59-84a4-43d1-ba44-c99d781b6896 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25371fa9f9264032bc796aed558ff250'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:35.790657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c18a0b59-84a4-43d1-ba44-c99d781b6896 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25371fa9f9264032bc796aed558ff250'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:35.810612 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1047/2113] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:35 2022] DELETE /identity/v3/users/25371fa9f9264032bc796aed558ff250 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:36.414666 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87bfb05-dce1-4c04-a87f-99ede906dfe2 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:36.418774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-a87bfb05-dce1-4c04-a87f-99ede906dfe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:36.446816 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87bfb05-dce1-4c04-a87f-99ede906dfe2 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:36.546717 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65da29c-ecac-4ec6-95d8-05be7dd621be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:36.548632 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65da29c-ecac-4ec6-95d8-05be7dd621be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:36.549285 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65da29c-ecac-4ec6-95d8-05be7dd621be admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:36.550230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65da29c-ecac-4ec6-95d8-05be7dd621be admin admin] PATH_INFO: `/v3/users/6122dacf4e9f4415b104fb7f7f2f1585` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:36.553298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65da29c-ecac-4ec6-95d8-05be7dd621be admin admin] RBAC: Authorizing `identity:delete_user(user_id=6122dacf4e9f4415b104fb7f7f2f1585)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:36.558998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65da29c-ecac-4ec6-95d8-05be7dd621be admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:36.642659 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-d65da29c-ecac-4ec6-95d8-05be7dd621be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6122dacf4e9f4415b104fb7f7f2f1585'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:36.656807 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-d65da29c-ecac-4ec6-95d8-05be7dd621be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6122dacf4e9f4415b104fb7f7f2f1585'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:36.698931 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a0b59-84a4-43d1-ba44-c99d781b6896 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:36.701409 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c18a0b59-84a4-43d1-ba44-c99d781b6896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:36.715554 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-d65da29c-ecac-4ec6-95d8-05be7dd621be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6122dacf4e9f4415b104fb7f7f2f1585'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:36.717266 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-d65da29c-ecac-4ec6-95d8-05be7dd621be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6122dacf4e9f4415b104fb7f7f2f1585'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:36.720270 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a0b59-84a4-43d1-ba44-c99d781b6896 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:36.739489 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705946e5-e5aa-4cc5-a473-d7c452bed88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:36.740953 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-d65da29c-ecac-4ec6-95d8-05be7dd621be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6122dacf4e9f4415b104fb7f7f2f1585'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:36.742130 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705946e5-e5aa-4cc5-a473-d7c452bed88f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:36.743990 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705946e5-e5aa-4cc5-a473-d7c452bed88f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:36.744472 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705946e5-e5aa-4cc5-a473-d7c452bed88f admin admin] PATH_INFO: `/v3/projects/dc7f9b96acb14aca890e684ef19028ba` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:36.753521 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705946e5-e5aa-4cc5-a473-d7c452bed88f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc7f9b96acb14aca890e684ef19028ba)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:36.764764 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1067/2114] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:36 2022] DELETE /identity/v3/users/6122dacf4e9f4415b104fb7f7f2f1585 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:36.765347 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705946e5-e5aa-4cc5-a473-d7c452bed88f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:36.767747 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26122246-6c7c-454d-8f84-6ffefb7fd98e None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:36.768695 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26122246-6c7c-454d-8f84-6ffefb7fd98e None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:36.769205 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26122246-6c7c-454d-8f84-6ffefb7fd98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:36.808096 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-26122246-6c7c-454d-8f84-6ffefb7fd98e None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:36.810762 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-26122246-6c7c-454d-8f84-6ffefb7fd98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:36.897384 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-705946e5-e5aa-4cc5-a473-d7c452bed88f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc7f9b96acb14aca890e684ef19028ba', 'request_id': 'req-705946e5-e5aa-4cc5-a473-d7c452bed88f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:36.955607 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-705946e5-e5aa-4cc5-a473-d7c452bed88f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc7f9b96acb14aca890e684ef19028ba', 'request_id': 'req-705946e5-e5aa-4cc5-a473-d7c452bed88f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:36.960936 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1068/2115] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:36.988376 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26122246-6c7c-454d-8f84-6ffefb7fd98e None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:36.992283 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-26122246-6c7c-454d-8f84-6ffefb7fd98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:36.994078 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-705946e5-e5aa-4cc5-a473-d7c452bed88f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc7f9b96acb14aca890e684ef19028ba', 'request_id': 'req-705946e5-e5aa-4cc5-a473-d7c452bed88f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:37.010864 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:37.047845 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-705946e5-e5aa-4cc5-a473-d7c452bed88f admin admin] The token cache is being invalidate because project dc7f9b96acb14aca890e684ef19028ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:37.048124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-705946e5-e5aa-4cc5-a473-d7c452bed88f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:37.050568 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-705946e5-e5aa-4cc5-a473-d7c452bed88f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:37.072234 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1048/2116] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:36 2022] DELETE /identity/v3/projects/dc7f9b96acb14aca890e684ef19028ba => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:37.075654 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50251a-7c50-414d-bf78-4a7b1174ec66 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:37.076139 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50251a-7c50-414d-bf78-4a7b1174ec66 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:37.076752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50251a-7c50-414d-bf78-4a7b1174ec66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:37.124783 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-5e50251a-7c50-414d-bf78-4a7b1174ec66 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:37.129067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5e50251a-7c50-414d-bf78-4a7b1174ec66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:37.191908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26122246-6c7c-454d-8f84-6ffefb7fd98e None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:37.335544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8e1a8-f070-47e7-847a-72bb4d28fa05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:37.337571 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8e1a8-f070-47e7-847a-72bb4d28fa05 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:37.338112 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8e1a8-f070-47e7-847a-72bb4d28fa05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:37.340015 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8e1a8-f070-47e7-847a-72bb4d28fa05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:37.345810 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-49c8e1a8-f070-47e7-847a-72bb4d28fa05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:37.346325 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1049/2117] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:13:37.358286 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a861aa-667a-4dd4-a2f5-acf39387e28d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:37.358672 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a861aa-667a-4dd4-a2f5-acf39387e28d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:37.359026 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a861aa-667a-4dd4-a2f5-acf39387e28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:37.368779 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8e1a8-f070-47e7-847a-72bb4d28fa05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:37.370830 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8e1a8-f070-47e7-847a-72bb4d28fa05 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:37.388664 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1069/2118] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:37.388974 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-35a861aa-667a-4dd4-a2f5-acf39387e28d None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:37.390552 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-35a861aa-667a-4dd4-a2f5-acf39387e28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:37.401996 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1050/2119] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:37.407356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8e1a8-f070-47e7-847a-72bb4d28fa05 service neutron] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:37.422587 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8e1a8-f070-47e7-847a-72bb4d28fa05 service neutron] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:37.433591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d27f7c-7a11-4838-972c-73708f6c4cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:37.434715 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d27f7c-7a11-4838-972c-73708f6c4cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:37.435177 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d27f7c-7a11-4838-972c-73708f6c4cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:37.435616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d27f7c-7a11-4838-972c-73708f6c4cc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:37.436583 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d27f7c-7a11-4838-972c-73708f6c4cc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:13:37.437178 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d27f7c-7a11-4838-972c-73708f6c4cc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:37.440044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d27f7c-7a11-4838-972c-73708f6c4cc9 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:37.459653 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1070/2120] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:13:37.465857 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a861aa-667a-4dd4-a2f5-acf39387e28d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:37.487528 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a861aa-667a-4dd4-a2f5-acf39387e28d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:37.500563 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958dff99-944e-4db3-b086-443b5822cd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:37.506570 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958dff99-944e-4db3-b086-443b5822cd11 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:37.506570 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958dff99-944e-4db3-b086-443b5822cd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:37.507088 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958dff99-944e-4db3-b086-443b5822cd11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:37.508633 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958dff99-944e-4db3-b086-443b5822cd11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:37.511457 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958dff99-944e-4db3-b086-443b5822cd11 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:37.564632 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1051/2121] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:37 2022] POST /identity/v3/projects => generated 371 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:37.570706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d27f7c-7a11-4838-972c-73708f6c4cc9 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:37.595164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d27f7c-7a11-4838-972c-73708f6c4cc9 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:37.772553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa0858-c3c8-4e3a-9bff-5bd14a078d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:37.774178 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa0858-c3c8-4e3a-9bff-5bd14a078d1f admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:37.774902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa0858-c3c8-4e3a-9bff-5bd14a078d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:37.780652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa0858-c3c8-4e3a-9bff-5bd14a078d1f admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:37.782071 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa0858-c3c8-4e3a-9bff-5bd14a078d1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:37.787901 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa0858-c3c8-4e3a-9bff-5bd14a078d1f admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:37.843325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1071/2122] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:37 2022] POST /identity/v3/users => generated 286 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:37.845234 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958dff99-944e-4db3-b086-443b5822cd11 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:37.868418 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958dff99-944e-4db3-b086-443b5822cd11 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:37.878697 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da91e4e-eb7e-473a-932f-4606886998c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:37.879750 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da91e4e-eb7e-473a-932f-4606886998c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:37.880040 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da91e4e-eb7e-473a-932f-4606886998c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:37.880319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da91e4e-eb7e-473a-932f-4606886998c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:37.881260 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da91e4e-eb7e-473a-932f-4606886998c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:37.883732 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da91e4e-eb7e-473a-932f-4606886998c6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:37.893367 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:37.909516 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1052/2123] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:37.914541 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa0858-c3c8-4e3a-9bff-5bd14a078d1f admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:37.934222 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa0858-c3c8-4e3a-9bff-5bd14a078d1f admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:37.942589 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238c8c1-15d8-499e-af11-8c411cbd5a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:37.943692 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238c8c1-15d8-499e-af11-8c411cbd5a71 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:37.944442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238c8c1-15d8-499e-af11-8c411cbd5a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:37.944932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238c8c1-15d8-499e-af11-8c411cbd5a71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:37.946510 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d238c8c1-15d8-499e-af11-8c411cbd5a71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:37.949524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d238c8c1-15d8-499e-af11-8c411cbd5a71 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:37.974630 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1072/2124] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:13:37.981863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da91e4e-eb7e-473a-932f-4606886998c6 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:38.000696 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da91e4e-eb7e-473a-932f-4606886998c6 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:38.008790 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877790a1-2e72-42de-ae39-48e772c1cb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:38.010079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877790a1-2e72-42de-ae39-48e772c1cb9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:38.010509 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877790a1-2e72-42de-ae39-48e772c1cb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:38.010896 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877790a1-2e72-42de-ae39-48e772c1cb9e admin admin] PATH_INFO: `/v3/projects/7b7043f964014dfc87f4f05a1412cec6/users/7d5ec1ea62414eb5b76cb9135b0515c2/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:38.040646 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877790a1-2e72-42de-ae39-48e772c1cb9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b7043f964014dfc87f4f05a1412cec6, user_id=7d5ec1ea62414eb5b76cb9135b0515c2, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:38.045243 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877790a1-2e72-42de-ae39-48e772c1cb9e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:38.065016 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1053/2125] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:37 2022] PUT /identity/v3/projects/7b7043f964014dfc87f4f05a1412cec6/users/7d5ec1ea62414eb5b76cb9135b0515c2/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:38.077029 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b3373-0a30-4af3-9b62-efba8d5faa24 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:38.077425 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b3373-0a30-4af3-9b62-efba8d5faa24 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:38.077779 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b3373-0a30-4af3-9b62-efba8d5faa24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:38.132360 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-358b3373-0a30-4af3-9b62-efba8d5faa24 None None] MFA Rules not processed for user `7d5ec1ea62414eb5b76cb9135b0515c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:38.134184 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-358b3373-0a30-4af3-9b62-efba8d5faa24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:38.178575 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:38.321239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1073/2126] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:38.348057 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877790a1-2e72-42de-ae39-48e772c1cb9e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:38.351214 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-877790a1-2e72-42de-ae39-48e772c1cb9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:38.362855 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b3373-0a30-4af3-9b62-efba8d5faa24 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:38.364876 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-358b3373-0a30-4af3-9b62-efba8d5faa24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:38.505497 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b3373-0a30-4af3-9b62-efba8d5faa24 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:38.520767 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d30bde-51df-473a-b168-e3f715f1375b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:38.521790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d30bde-51df-473a-b168-e3f715f1375b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:38.522150 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d30bde-51df-473a-b168-e3f715f1375b admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:38.522416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d30bde-51df-473a-b168-e3f715f1375b admin admin] PATH_INFO: `/v3/users/136d02137cb8458ab66659512fd11f2a` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:38.526127 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d30bde-51df-473a-b168-e3f715f1375b admin admin] RBAC: Authorizing `identity:delete_user(user_id=136d02137cb8458ab66659512fd11f2a)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:38.528171 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d30bde-51df-473a-b168-e3f715f1375b admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:38.537310 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877790a1-2e72-42de-ae39-48e772c1cb9e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:38.547368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf42df7-fbdd-41f7-bde3-39239a68857f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f11beca9c5148eea4cc45f8dc89dbda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:38.548454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf42df7-fbdd-41f7-bde3-39239a68857f service glance] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:38.548760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf42df7-fbdd-41f7-bde3-39239a68857f service glance] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:38.549082 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf42df7-fbdd-41f7-bde3-39239a68857f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:38.564854 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf42df7-fbdd-41f7-bde3-39239a68857f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:38.567498 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf42df7-fbdd-41f7-bde3-39239a68857f service glance] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:38.583040 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1054/2127] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:38.657550 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-b1d30bde-51df-473a-b168-e3f715f1375b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '136d02137cb8458ab66659512fd11f2a'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:38.686870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-b1d30bde-51df-473a-b168-e3f715f1375b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '136d02137cb8458ab66659512fd11f2a'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:38.707772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf42df7-fbdd-41f7-bde3-39239a68857f service glance] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:38.709601 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8cf42df7-fbdd-41f7-bde3-39239a68857f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:38.746518 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf42df7-fbdd-41f7-bde3-39239a68857f service glance] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:38.754793 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-b1d30bde-51df-473a-b168-e3f715f1375b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '136d02137cb8458ab66659512fd11f2a'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:38.756431 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-b1d30bde-51df-473a-b168-e3f715f1375b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '136d02137cb8458ab66659512fd11f2a'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:38.767082 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2abf72-05cd-40f9-9935-b461dd00a67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:38.771734 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2abf72-05cd-40f9-9935-b461dd00a67e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:38.772138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2abf72-05cd-40f9-9935-b461dd00a67e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:38.772512 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2abf72-05cd-40f9-9935-b461dd00a67e admin admin] PATH_INFO: `/v3/users/7ce0d7fca12a4e2aa11f0c193624c6f7` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:38.784680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2abf72-05cd-40f9-9935-b461dd00a67e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ce0d7fca12a4e2aa11f0c193624c6f7)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:38.785825 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-b1d30bde-51df-473a-b168-e3f715f1375b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '136d02137cb8458ab66659512fd11f2a'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:38.790395 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2abf72-05cd-40f9-9935-b461dd00a67e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:38.818427 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1074/2128] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:38 2022] DELETE /identity/v3/users/136d02137cb8458ab66659512fd11f2a => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:38.824871 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3e211-1b55-4983-a611-0610ac2f89a1 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:38.825189 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3e211-1b55-4983-a611-0610ac2f89a1 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:38.825476 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3e211-1b55-4983-a611-0610ac2f89a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:38.901645 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-5ff3e211-1b55-4983-a611-0610ac2f89a1 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:38.903429 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-5ff3e211-1b55-4983-a611-0610ac2f89a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:38.921213 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7f2abf72-05cd-40f9-9935-b461dd00a67e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ce0d7fca12a4e2aa11f0c193624c6f7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:38.954990 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7f2abf72-05cd-40f9-9935-b461dd00a67e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ce0d7fca12a4e2aa11f0c193624c6f7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:38.970839 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7f2abf72-05cd-40f9-9935-b461dd00a67e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ce0d7fca12a4e2aa11f0c193624c6f7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:38.993167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:39.008005 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7f2abf72-05cd-40f9-9935-b461dd00a67e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ce0d7fca12a4e2aa11f0c193624c6f7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:39.022367 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7f2abf72-05cd-40f9-9935-b461dd00a67e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ce0d7fca12a4e2aa11f0c193624c6f7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:39.067514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1055/2129] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:38 2022] DELETE /identity/v3/users/7ce0d7fca12a4e2aa11f0c193624c6f7 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:39.097989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1075/2130] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:13:39.126721 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2abf72-05cd-40f9-9935-b461dd00a67e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:39.130506 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7f2abf72-05cd-40f9-9935-b461dd00a67e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:39.359097 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2abf72-05cd-40f9-9935-b461dd00a67e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:39.366839 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f5143-aaa5-4917-9f0a-e0aadfe48b8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:39.368059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f5143-aaa5-4917-9f0a-e0aadfe48b8b service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:39.368571 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f5143-aaa5-4917-9f0a-e0aadfe48b8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:39.369061 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f5143-aaa5-4917-9f0a-e0aadfe48b8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:39.388194 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:39.421452 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3e211-1b55-4983-a611-0610ac2f89a1 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:39.423991 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-5ff3e211-1b55-4983-a611-0610ac2f89a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:39.463608 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3e211-1b55-4983-a611-0610ac2f89a1 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:39.634772 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612a2ee-6193-4bd1-9fe6-c7cd0748adb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:39.636449 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f5143-aaa5-4917-9f0a-e0aadfe48b8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:39.637096 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612a2ee-6193-4bd1-9fe6-c7cd0748adb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:39.637449 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612a2ee-6193-4bd1-9fe6-c7cd0748adb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:39.637788 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612a2ee-6193-4bd1-9fe6-c7cd0748adb6 admin admin] PATH_INFO: `/v3/projects/591ca27a4da1433d8a620709b6025ffe` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:39.637901 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f5143-aaa5-4917-9f0a-e0aadfe48b8b service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:39.639508 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8612a2ee-6193-4bd1-9fe6-c7cd0748adb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=591ca27a4da1433d8a620709b6025ffe)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:39.642536 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8612a2ee-6193-4bd1-9fe6-c7cd0748adb6 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:39.668582 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1056/2131] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:39.670918 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f5143-aaa5-4917-9f0a-e0aadfe48b8b service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:39.676346 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-526f5143-aaa5-4917-9f0a-e0aadfe48b8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:39.711611 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f5143-aaa5-4917-9f0a-e0aadfe48b8b service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:39.744004 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-8612a2ee-6193-4bd1-9fe6-c7cd0748adb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '591ca27a4da1433d8a620709b6025ffe', 'request_id': 'req-8612a2ee-6193-4bd1-9fe6-c7cd0748adb6'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:39.785245 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-8612a2ee-6193-4bd1-9fe6-c7cd0748adb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '591ca27a4da1433d8a620709b6025ffe', 'request_id': 'req-8612a2ee-6193-4bd1-9fe6-c7cd0748adb6'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:39.801368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-8612a2ee-6193-4bd1-9fe6-c7cd0748adb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '591ca27a4da1433d8a620709b6025ffe', 'request_id': 'req-8612a2ee-6193-4bd1-9fe6-c7cd0748adb6'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:39.830623 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-8612a2ee-6193-4bd1-9fe6-c7cd0748adb6 admin admin] The token cache is being invalidate because project 591ca27a4da1433d8a620709b6025ffe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:39.833180 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-8612a2ee-6193-4bd1-9fe6-c7cd0748adb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:39.835364 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-8612a2ee-6193-4bd1-9fe6-c7cd0748adb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:39.838897 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1076/2132] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:39 2022] DELETE /identity/v3/projects/591ca27a4da1433d8a620709b6025ffe => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:39.844929 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612a2ee-6193-4bd1-9fe6-c7cd0748adb6 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:39.848743 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-8612a2ee-6193-4bd1-9fe6-c7cd0748adb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:40.018684 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79985dd1-aa17-4be9-a116-60420ebb01f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:40.020026 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79985dd1-aa17-4be9-a116-60420ebb01f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:40.020535 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79985dd1-aa17-4be9-a116-60420ebb01f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:40.025547 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79985dd1-aa17-4be9-a116-60420ebb01f1 admin admin] PATH_INFO: `/v3/projects/2c4305bcb3af4a92b109f25f8f0bb382` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:40.034023 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79985dd1-aa17-4be9-a116-60420ebb01f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c4305bcb3af4a92b109f25f8f0bb382)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:40.046934 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612a2ee-6193-4bd1-9fe6-c7cd0748adb6 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:40.050883 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79985dd1-aa17-4be9-a116-60420ebb01f1 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:40.065211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341070b3-9b44-4f33-86d9-87f406467039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:40.067452 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341070b3-9b44-4f33-86d9-87f406467039 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:40.068209 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341070b3-9b44-4f33-86d9-87f406467039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:40.069995 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341070b3-9b44-4f33-86d9-87f406467039 admin admin] PATH_INFO: `/v3/users/7a63d66ffe664428b381a708d11e4a3f` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:40.072994 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341070b3-9b44-4f33-86d9-87f406467039 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a63d66ffe664428b381a708d11e4a3f)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:40.078722 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341070b3-9b44-4f33-86d9-87f406467039 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:40.086502 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:40.174338 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-79985dd1-aa17-4be9-a116-60420ebb01f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c4305bcb3af4a92b109f25f8f0bb382', 'request_id': 'req-79985dd1-aa17-4be9-a116-60420ebb01f1'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:40.184824 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-79985dd1-aa17-4be9-a116-60420ebb01f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c4305bcb3af4a92b109f25f8f0bb382', 'request_id': 'req-79985dd1-aa17-4be9-a116-60420ebb01f1'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:40.205070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-79985dd1-aa17-4be9-a116-60420ebb01f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c4305bcb3af4a92b109f25f8f0bb382', 'request_id': 'req-79985dd1-aa17-4be9-a116-60420ebb01f1'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:40.226793 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-341070b3-9b44-4f33-86d9-87f406467039 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a63d66ffe664428b381a708d11e4a3f'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:40.232214 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-79985dd1-aa17-4be9-a116-60420ebb01f1 admin admin] The token cache is being invalidate because project 2c4305bcb3af4a92b109f25f8f0bb382 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:40.232577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-79985dd1-aa17-4be9-a116-60420ebb01f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:40.233842 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-79985dd1-aa17-4be9-a116-60420ebb01f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:40.236077 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1057/2133] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:39 2022] DELETE /identity/v3/projects/2c4305bcb3af4a92b109f25f8f0bb382 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:40.243444 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79985dd1-aa17-4be9-a116-60420ebb01f1 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:40.248210 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-79985dd1-aa17-4be9-a116-60420ebb01f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:40.264625 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-341070b3-9b44-4f33-86d9-87f406467039 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a63d66ffe664428b381a708d11e4a3f'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:40.324241 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-341070b3-9b44-4f33-86d9-87f406467039 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a63d66ffe664428b381a708d11e4a3f'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:40.326506 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-341070b3-9b44-4f33-86d9-87f406467039 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a63d66ffe664428b381a708d11e4a3f'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:40.343124 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-341070b3-9b44-4f33-86d9-87f406467039 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a63d66ffe664428b381a708d11e4a3f'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:40.366019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:40.370289 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1077/2134] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:39 2022] DELETE /identity/v3/users/7a63d66ffe664428b381a708d11e4a3f => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:40.457472 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79985dd1-aa17-4be9-a116-60420ebb01f1 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:40.624738 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a39e2a-1d6b-41f1-b12d-aa29bbe42768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:40.629018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341070b3-9b44-4f33-86d9-87f406467039 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:40.631102 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-341070b3-9b44-4f33-86d9-87f406467039 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:40.636784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a39e2a-1d6b-41f1-b12d-aa29bbe42768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:40.643111 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a39e2a-1d6b-41f1-b12d-aa29bbe42768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:40.643790 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a39e2a-1d6b-41f1-b12d-aa29bbe42768 admin admin] PATH_INFO: `/v3/users/6de40790b903490dbcc29e10e48d0ac4` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:40.653816 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a39e2a-1d6b-41f1-b12d-aa29bbe42768 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6de40790b903490dbcc29e10e48d0ac4)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:40.663039 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a39e2a-1d6b-41f1-b12d-aa29bbe42768 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:40.673145 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341070b3-9b44-4f33-86d9-87f406467039 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:40.685304 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d08f46-02be-4e62-982b-cf64f32c2cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:40.686376 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d08f46-02be-4e62-982b-cf64f32c2cb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:40.686661 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d08f46-02be-4e62-982b-cf64f32c2cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:40.687083 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d08f46-02be-4e62-982b-cf64f32c2cb7 admin admin] PATH_INFO: `/v3/projects/29303a4700e340c6a90f0233af0fbfac` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:40.688987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d08f46-02be-4e62-982b-cf64f32c2cb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29303a4700e340c6a90f0233af0fbfac)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:40.703738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d08f46-02be-4e62-982b-cf64f32c2cb7 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:40.790743 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-92a39e2a-1d6b-41f1-b12d-aa29bbe42768 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6de40790b903490dbcc29e10e48d0ac4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:40.813862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-92a39e2a-1d6b-41f1-b12d-aa29bbe42768 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6de40790b903490dbcc29e10e48d0ac4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:40.819676 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-68d08f46-02be-4e62-982b-cf64f32c2cb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29303a4700e340c6a90f0233af0fbfac', 'request_id': 'req-68d08f46-02be-4e62-982b-cf64f32c2cb7'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:40.824208 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-92a39e2a-1d6b-41f1-b12d-aa29bbe42768 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6de40790b903490dbcc29e10e48d0ac4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:40.835569 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-68d08f46-02be-4e62-982b-cf64f32c2cb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29303a4700e340c6a90f0233af0fbfac', 'request_id': 'req-68d08f46-02be-4e62-982b-cf64f32c2cb7'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:40.836424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-92a39e2a-1d6b-41f1-b12d-aa29bbe42768 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6de40790b903490dbcc29e10e48d0ac4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:40.837276 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-92a39e2a-1d6b-41f1-b12d-aa29bbe42768 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6de40790b903490dbcc29e10e48d0ac4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:40.861072 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1058/2135] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:40 2022] DELETE /identity/v3/users/6de40790b903490dbcc29e10e48d0ac4 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:40.882109 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-68d08f46-02be-4e62-982b-cf64f32c2cb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29303a4700e340c6a90f0233af0fbfac', 'request_id': 'req-68d08f46-02be-4e62-982b-cf64f32c2cb7'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:40.923253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-68d08f46-02be-4e62-982b-cf64f32c2cb7 admin admin] The token cache is being invalidate because project 29303a4700e340c6a90f0233af0fbfac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:40.923564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-68d08f46-02be-4e62-982b-cf64f32c2cb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:40.934776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-68d08f46-02be-4e62-982b-cf64f32c2cb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:40.944187 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37124daa-7c97-435c-be82-d04aed97e9ed None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:40.944595 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37124daa-7c97-435c-be82-d04aed97e9ed None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:40.944891 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37124daa-7c97-435c-be82-d04aed97e9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:40.947151 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1078/2136] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:40 2022] DELETE /identity/v3/projects/29303a4700e340c6a90f0233af0fbfac => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:40.983820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-37124daa-7c97-435c-be82-d04aed97e9ed None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:40.986435 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-37124daa-7c97-435c-be82-d04aed97e9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:41.118582 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:41.140503 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1059/2137] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:13:41.149434 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91e05d-0f94-4ba3-973f-dc0d02d505f8 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:41.149849 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91e05d-0f94-4ba3-973f-dc0d02d505f8 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:41.151191 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91e05d-0f94-4ba3-973f-dc0d02d505f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:41.176302 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-5a91e05d-0f94-4ba3-973f-dc0d02d505f8 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:41.178104 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-5a91e05d-0f94-4ba3-973f-dc0d02d505f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:41.194510 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1079/2138] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:41.199901 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37124daa-7c97-435c-be82-d04aed97e9ed None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:41.225348 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37124daa-7c97-435c-be82-d04aed97e9ed None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:41.237394 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95d462-696a-40ec-a569-625fcfcc0eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:41.238304 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95d462-696a-40ec-a569-625fcfcc0eda admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:41.238788 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95d462-696a-40ec-a569-625fcfcc0eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:41.239627 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95d462-696a-40ec-a569-625fcfcc0eda admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:41.240698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95d462-696a-40ec-a569-625fcfcc0eda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:13:41.241256 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95d462-696a-40ec-a569-625fcfcc0eda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:41.244726 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95d462-696a-40ec-a569-625fcfcc0eda admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:41.250733 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a91e05d-0f94-4ba3-973f-dc0d02d505f8 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:41.254726 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-5a91e05d-0f94-4ba3-973f-dc0d02d505f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:41.277999 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1060/2139] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:13:41.283212 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95d462-696a-40ec-a569-625fcfcc0eda admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:41.297535 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a91e05d-0f94-4ba3-973f-dc0d02d505f8 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:41.301649 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95d462-696a-40ec-a569-625fcfcc0eda admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:41.308335 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372fb53-d5ce-48cd-9908-cc8067662375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:41.313008 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bda2a8-886b-43a7-b420-db147e82f79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:41.313873 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bda2a8-886b-43a7-b420-db147e82f79f admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:41.314327 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bda2a8-886b-43a7-b420-db147e82f79f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:41.315617 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bda2a8-886b-43a7-b420-db147e82f79f admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:41.317720 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bda2a8-886b-43a7-b420-db147e82f79f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:41.317912 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372fb53-d5ce-48cd-9908-cc8067662375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:41.317912 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372fb53-d5ce-48cd-9908-cc8067662375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:41.317912 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372fb53-d5ce-48cd-9908-cc8067662375 admin admin] PATH_INFO: `/v3/projects/09e52c177a144479a203c60a4ce16cdd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:41.319712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372fb53-d5ce-48cd-9908-cc8067662375 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09e52c177a144479a203c60a4ce16cdd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:41.323421 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bda2a8-886b-43a7-b420-db147e82f79f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:41.325442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372fb53-d5ce-48cd-9908-cc8067662375 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:41.399618 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1061/2140] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:41 2022] POST /identity/v3/projects => generated 397 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:41.402315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bda2a8-886b-43a7-b420-db147e82f79f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:41.460845 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-e372fb53-d5ce-48cd-9908-cc8067662375 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09e52c177a144479a203c60a4ce16cdd', 'request_id': 'req-e372fb53-d5ce-48cd-9908-cc8067662375'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:41.481593 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=75,keystone:DELETE=23,keystone:INSERT=2 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:41.483681 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-e372fb53-d5ce-48cd-9908-cc8067662375 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09e52c177a144479a203c60a4ce16cdd', 'request_id': 'req-e372fb53-d5ce-48cd-9908-cc8067662375'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:41.496945 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-e372fb53-d5ce-48cd-9908-cc8067662375 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09e52c177a144479a203c60a4ce16cdd', 'request_id': 'req-e372fb53-d5ce-48cd-9908-cc8067662375'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:41.537224 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-e372fb53-d5ce-48cd-9908-cc8067662375 admin admin] The token cache is being invalidate because project 09e52c177a144479a203c60a4ce16cdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:41.538176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-e372fb53-d5ce-48cd-9908-cc8067662375 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:41.539368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-e372fb53-d5ce-48cd-9908-cc8067662375 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:41.542278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1080/2141] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:41 2022] DELETE /identity/v3/projects/09e52c177a144479a203c60a4ce16cdd => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:41.545291 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372fb53-d5ce-48cd-9908-cc8067662375 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:41.546503 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-e372fb53-d5ce-48cd-9908-cc8067662375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:41.560298 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bda2a8-886b-43a7-b420-db147e82f79f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:41.650154 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372fb53-d5ce-48cd-9908-cc8067662375 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:41.664844 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a55cba-17dc-4f1b-8926-2034f22be6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:41.666136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a55cba-17dc-4f1b-8926-2034f22be6b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:41.667075 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a55cba-17dc-4f1b-8926-2034f22be6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:41.667356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a55cba-17dc-4f1b-8926-2034f22be6b9 admin admin] PATH_INFO: `/v3/projects/80d4a4c1f4ad475b95f54ba5971736ae` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:41.669763 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a55cba-17dc-4f1b-8926-2034f22be6b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80d4a4c1f4ad475b95f54ba5971736ae)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:41.672421 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a55cba-17dc-4f1b-8926-2034f22be6b9 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:41.684113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30261c1d-fcea-4d2b-8e2b-b155fdcfeaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:41.684914 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30261c1d-fcea-4d2b-8e2b-b155fdcfeaa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:41.685289 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30261c1d-fcea-4d2b-8e2b-b155fdcfeaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:41.685634 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30261c1d-fcea-4d2b-8e2b-b155fdcfeaa4 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:41.686467 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30261c1d-fcea-4d2b-8e2b-b155fdcfeaa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:41.689190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30261c1d-fcea-4d2b-8e2b-b155fdcfeaa4 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:41.725256 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1062/2142] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:41 2022] POST /identity/v3/users => generated 299 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:41.725854 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30261c1d-fcea-4d2b-8e2b-b155fdcfeaa4 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:41.727699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-30261c1d-fcea-4d2b-8e2b-b155fdcfeaa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:41.748797 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30261c1d-fcea-4d2b-8e2b-b155fdcfeaa4 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:41.768768 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-c6a55cba-17dc-4f1b-8926-2034f22be6b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80d4a4c1f4ad475b95f54ba5971736ae', 'request_id': 'req-c6a55cba-17dc-4f1b-8926-2034f22be6b9'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:41.793575 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-c6a55cba-17dc-4f1b-8926-2034f22be6b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80d4a4c1f4ad475b95f54ba5971736ae', 'request_id': 'req-c6a55cba-17dc-4f1b-8926-2034f22be6b9'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:41.811433 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-c6a55cba-17dc-4f1b-8926-2034f22be6b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80d4a4c1f4ad475b95f54ba5971736ae', 'request_id': 'req-c6a55cba-17dc-4f1b-8926-2034f22be6b9'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:41.848639 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-c6a55cba-17dc-4f1b-8926-2034f22be6b9 admin admin] The token cache is being invalidate because project 80d4a4c1f4ad475b95f54ba5971736ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:41.848950 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-c6a55cba-17dc-4f1b-8926-2034f22be6b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:41.849747 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-c6a55cba-17dc-4f1b-8926-2034f22be6b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:41.852218 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1081/2143] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:41 2022] DELETE /identity/v3/projects/80d4a4c1f4ad475b95f54ba5971736ae => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:41.858206 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38688a-8f3f-40b1-bc0a-b05bc2349db3 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:41.858672 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38688a-8f3f-40b1-bc0a-b05bc2349db3 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:41.859153 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38688a-8f3f-40b1-bc0a-b05bc2349db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:41.876077 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514737be-0000-44e8-925c-d27c4c5b5fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:41.877183 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514737be-0000-44e8-925c-d27c4c5b5fba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:41.878398 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514737be-0000-44e8-925c-d27c4c5b5fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:41.878885 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514737be-0000-44e8-925c-d27c4c5b5fba admin admin] PATH_INFO: `/v3/users/4d48e41e19584c5c8773c9f445355ff2` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:41.881352 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514737be-0000-44e8-925c-d27c4c5b5fba admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d48e41e19584c5c8773c9f445355ff2)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:41.884366 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514737be-0000-44e8-925c-d27c4c5b5fba admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:41.887472 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:41.890956 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-ae38688a-8f3f-40b1-bc0a-b05bc2349db3 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:41.893099 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-ae38688a-8f3f-40b1-bc0a-b05bc2349db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:41.910151 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1082/2144] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:41.911318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae38688a-8f3f-40b1-bc0a-b05bc2349db3 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:41.914305 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-ae38688a-8f3f-40b1-bc0a-b05bc2349db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:41.927491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae38688a-8f3f-40b1-bc0a-b05bc2349db3 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:41.933419 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0bd605-51f3-487e-8d7d-d91f71739993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:41.934171 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0bd605-51f3-487e-8d7d-d91f71739993 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:41.934459 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0bd605-51f3-487e-8d7d-d91f71739993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:41.935019 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0bd605-51f3-487e-8d7d-d91f71739993 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:41.935817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0bd605-51f3-487e-8d7d-d91f71739993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:41.937900 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0bd605-51f3-487e-8d7d-d91f71739993 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:41.954758 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1083/2145] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:41.956412 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0bd605-51f3-487e-8d7d-d91f71739993 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:41.963002 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-2c0bd605-51f3-487e-8d7d-d91f71739993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:41.964941 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-514737be-0000-44e8-925c-d27c4c5b5fba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d48e41e19584c5c8773c9f445355ff2'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:41.982383 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0bd605-51f3-487e-8d7d-d91f71739993 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:41.985148 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-514737be-0000-44e8-925c-d27c4c5b5fba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d48e41e19584c5c8773c9f445355ff2'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:41.988881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a96448-4410-4b82-8945-c1170586ded9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:41.990889 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a96448-4410-4b82-8945-c1170586ded9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:41.991613 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a96448-4410-4b82-8945-c1170586ded9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:41.992219 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a96448-4410-4b82-8945-c1170586ded9 admin admin] PATH_INFO: `/v3/users/1ca8abb561864565a4c70a2fb8eaa0d6` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:41.994587 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a96448-4410-4b82-8945-c1170586ded9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ca8abb561864565a4c70a2fb8eaa0d6)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:41.995327 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-514737be-0000-44e8-925c-d27c4c5b5fba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d48e41e19584c5c8773c9f445355ff2'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:41.998542 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a96448-4410-4b82-8945-c1170586ded9 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:42.008246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-514737be-0000-44e8-925c-d27c4c5b5fba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d48e41e19584c5c8773c9f445355ff2'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:42.009189 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-514737be-0000-44e8-925c-d27c4c5b5fba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d48e41e19584c5c8773c9f445355ff2'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:42.018119 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1063/2146] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:41 2022] DELETE /identity/v3/users/4d48e41e19584c5c8773c9f445355ff2 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:42.024420 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048bb81-f585-4fd0-81cb-e4d177a0202a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:42.024941 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048bb81-f585-4fd0-81cb-e4d177a0202a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:42.025469 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048bb81-f585-4fd0-81cb-e4d177a0202a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:42.061915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4048bb81-f585-4fd0-81cb-e4d177a0202a None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:42.064215 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4048bb81-f585-4fd0-81cb-e4d177a0202a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:42.082878 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-51a96448-4410-4b82-8945-c1170586ded9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ca8abb561864565a4c70a2fb8eaa0d6'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:42.088938 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=83,keystone:INSERT=2,keystone:DELETE=15 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:42.095128 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-51a96448-4410-4b82-8945-c1170586ded9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ca8abb561864565a4c70a2fb8eaa0d6'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:42.141732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-51a96448-4410-4b82-8945-c1170586ded9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ca8abb561864565a4c70a2fb8eaa0d6'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:42.143033 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-51a96448-4410-4b82-8945-c1170586ded9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ca8abb561864565a4c70a2fb8eaa0d6'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:42.158646 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-51a96448-4410-4b82-8945-c1170586ded9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ca8abb561864565a4c70a2fb8eaa0d6'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:42.172425 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1084/2147] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:41 2022] DELETE /identity/v3/users/1ca8abb561864565a4c70a2fb8eaa0d6 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:42.172988 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a96448-4410-4b82-8945-c1170586ded9 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:42.174884 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-51a96448-4410-4b82-8945-c1170586ded9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:42.272084 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1064/2148] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:42.277458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4048bb81-f585-4fd0-81cb-e4d177a0202a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:42.289786 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4048bb81-f585-4fd0-81cb-e4d177a0202a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:42.378394 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a96448-4410-4b82-8945-c1170586ded9 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:42.394037 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e1dd9-3160-4e7d-8c7a-d10ca4aec166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:42.394920 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e1dd9-3160-4e7d-8c7a-d10ca4aec166 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:42.395865 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e1dd9-3160-4e7d-8c7a-d10ca4aec166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:42.395865 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e1dd9-3160-4e7d-8c7a-d10ca4aec166 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:42.396377 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e1dd9-3160-4e7d-8c7a-d10ca4aec166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:42.399765 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e1dd9-3160-4e7d-8c7a-d10ca4aec166 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:42.410857 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1085/2149] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:42.421766 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e1dd9-3160-4e7d-8c7a-d10ca4aec166 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:42.433230 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d48d07-2b1e-4232-a9cf-34137f46a900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:42.434418 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d48d07-2b1e-4232-a9cf-34137f46a900 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:42.434833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d48d07-2b1e-4232-a9cf-34137f46a900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:42.435253 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d48d07-2b1e-4232-a9cf-34137f46a900 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:42.435593 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e1dd9-3160-4e7d-8c7a-d10ca4aec166 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:42.436356 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d48d07-2b1e-4232-a9cf-34137f46a900 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:13:42.436825 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d48d07-2b1e-4232-a9cf-34137f46a900 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:42.439454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d48d07-2b1e-4232-a9cf-34137f46a900 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:42.443097 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf246103-2be7-4284-b61f-bb3be4db1663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:42.444296 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf246103-2be7-4284-b61f-bb3be4db1663 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:42.444709 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf246103-2be7-4284-b61f-bb3be4db1663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:42.444995 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf246103-2be7-4284-b61f-bb3be4db1663 admin admin] PATH_INFO: `/v3/projects/a9843ebd86dc49608e83ce0ad80c678e/users/e31cd745f6e04f1f998f57ab6bbe934f/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:42.471193 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1065/2150] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:13:42.476245 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf246103-2be7-4284-b61f-bb3be4db1663 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9843ebd86dc49608e83ce0ad80c678e, user_id=e31cd745f6e04f1f998f57ab6bbe934f, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:42.482679 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d48d07-2b1e-4232-a9cf-34137f46a900 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:42.482852 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf246103-2be7-4284-b61f-bb3be4db1663 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:42.484630 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-12d48d07-2b1e-4232-a9cf-34137f46a900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:42.517288 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1086/2151] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:42 2022] PUT /identity/v3/projects/a9843ebd86dc49608e83ce0ad80c678e/users/e31cd745f6e04f1f998f57ab6bbe934f/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:42.524555 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf246103-2be7-4284-b61f-bb3be4db1663 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:42.541950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d48d07-2b1e-4232-a9cf-34137f46a900 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:42.562218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:42.665571 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cba640-fd9e-4367-9607-17d71789bcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:42.667036 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cba640-fd9e-4367-9607-17d71789bcef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:42.667323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cba640-fd9e-4367-9607-17d71789bcef admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:42.667604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cba640-fd9e-4367-9607-17d71789bcef admin admin] PATH_INFO: `/v3/projects/4e4bfae59d38461c839fc9a35eb4d6cf` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:42.668997 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cba640-fd9e-4367-9607-17d71789bcef admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e4bfae59d38461c839fc9a35eb4d6cf)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:42.671099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cba640-fd9e-4367-9607-17d71789bcef admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:42.680277 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf246103-2be7-4284-b61f-bb3be4db1663 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:42.689513 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b293f1d-7468-4f4d-89cd-04c12761507f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:42.690705 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b293f1d-7468-4f4d-89cd-04c12761507f admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:42.691189 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b293f1d-7468-4f4d-89cd-04c12761507f admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:42.691706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b293f1d-7468-4f4d-89cd-04c12761507f admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:42.692897 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b293f1d-7468-4f4d-89cd-04c12761507f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:42.696187 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b293f1d-7468-4f4d-89cd-04c12761507f admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:42.749056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1087/2152] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:42 2022] POST /identity/v3/projects => generated 377 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:42.751437 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5117ee-1a3f-4aba-8afe-75c7c76a3842 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:42.751903 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5117ee-1a3f-4aba-8afe-75c7c76a3842 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:42.766599 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5117ee-1a3f-4aba-8afe-75c7c76a3842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:42.790626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-66cba640-fd9e-4367-9607-17d71789bcef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e4bfae59d38461c839fc9a35eb4d6cf', 'request_id': 'req-66cba640-fd9e-4367-9607-17d71789bcef'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:42.802438 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-66cba640-fd9e-4367-9607-17d71789bcef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e4bfae59d38461c839fc9a35eb4d6cf', 'request_id': 'req-66cba640-fd9e-4367-9607-17d71789bcef'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:42.816978 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-66cba640-fd9e-4367-9607-17d71789bcef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e4bfae59d38461c839fc9a35eb4d6cf', 'request_id': 'req-66cba640-fd9e-4367-9607-17d71789bcef'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:42.820635 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-cd5117ee-1a3f-4aba-8afe-75c7c76a3842 None None] MFA Rules not processed for user `e31cd745f6e04f1f998f57ab6bbe934f`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:42.822132 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-cd5117ee-1a3f-4aba-8afe-75c7c76a3842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:42.837498 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-66cba640-fd9e-4367-9607-17d71789bcef admin admin] The token cache is being invalidate because project 4e4bfae59d38461c839fc9a35eb4d6cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:42.838890 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-66cba640-fd9e-4367-9607-17d71789bcef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:42.839588 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-66cba640-fd9e-4367-9607-17d71789bcef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:42.842181 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:42.842503 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1066/2153] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:42 2022] DELETE /identity/v3/projects/4e4bfae59d38461c839fc9a35eb4d6cf => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:42.843864 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cba640-fd9e-4367-9607-17d71789bcef admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:42.846139 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-66cba640-fd9e-4367-9607-17d71789bcef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:42.940009 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1088/2154] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:42.941603 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5117ee-1a3f-4aba-8afe-75c7c76a3842 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:42.947669 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cba640-fd9e-4367-9607-17d71789bcef admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:42.950141 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-cd5117ee-1a3f-4aba-8afe-75c7c76a3842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:43.015657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:43.039954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b80627-8dd4-4e9a-a2fd-a1ce81644d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:43.042014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b80627-8dd4-4e9a-a2fd-a1ce81644d1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:43.043079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b80627-8dd4-4e9a-a2fd-a1ce81644d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:43.043763 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b80627-8dd4-4e9a-a2fd-a1ce81644d1c admin admin] PATH_INFO: `/v3/projects/f13bd6240aa34f0e84612c1a77e4715b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:43.045956 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b80627-8dd4-4e9a-a2fd-a1ce81644d1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f13bd6240aa34f0e84612c1a77e4715b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:43.049298 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b80627-8dd4-4e9a-a2fd-a1ce81644d1c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:43.086009 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5117ee-1a3f-4aba-8afe-75c7c76a3842 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:43.092733 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b795d14-6877-44fd-8638-356cc6d18e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:43.093425 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b795d14-6877-44fd-8638-356cc6d18e65 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:43.093715 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b795d14-6877-44fd-8638-356cc6d18e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:43.094018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b795d14-6877-44fd-8638-356cc6d18e65 admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:43.094731 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b795d14-6877-44fd-8638-356cc6d18e65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:43.096710 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b795d14-6877-44fd-8638-356cc6d18e65 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:43.129040 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1089/2155] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:42 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:43.131703 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e2f8f-9e64-4122-8df9-b7a12c435f68 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:43.132231 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e2f8f-9e64-4122-8df9-b7a12c435f68 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:43.132950 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e2f8f-9e64-4122-8df9-b7a12c435f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:43.140174 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-12b80627-8dd4-4e9a-a2fd-a1ce81644d1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f13bd6240aa34f0e84612c1a77e4715b', 'request_id': 'req-12b80627-8dd4-4e9a-a2fd-a1ce81644d1c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:43.149135 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-12b80627-8dd4-4e9a-a2fd-a1ce81644d1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f13bd6240aa34f0e84612c1a77e4715b', 'request_id': 'req-12b80627-8dd4-4e9a-a2fd-a1ce81644d1c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:43.167685 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-12b80627-8dd4-4e9a-a2fd-a1ce81644d1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f13bd6240aa34f0e84612c1a77e4715b', 'request_id': 'req-12b80627-8dd4-4e9a-a2fd-a1ce81644d1c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:43.175722 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-045e2f8f-9e64-4122-8df9-b7a12c435f68 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:43.176939 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-045e2f8f-9e64-4122-8df9-b7a12c435f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:43.183857 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-12b80627-8dd4-4e9a-a2fd-a1ce81644d1c admin admin] The token cache is being invalidate because project f13bd6240aa34f0e84612c1a77e4715b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:43.184150 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-12b80627-8dd4-4e9a-a2fd-a1ce81644d1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:43.184848 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-12b80627-8dd4-4e9a-a2fd-a1ce81644d1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:43.186642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1067/2156] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:42 2022] DELETE /identity/v3/projects/f13bd6240aa34f0e84612c1a77e4715b => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:43.187540 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b80627-8dd4-4e9a-a2fd-a1ce81644d1c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:43.192337 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-12b80627-8dd4-4e9a-a2fd-a1ce81644d1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:43.284986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b80627-8dd4-4e9a-a2fd-a1ce81644d1c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:43.292932 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b974b-014d-4621-8fb4-5944d075a316 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:43.293724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b974b-014d-4621-8fb4-5944d075a316 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:43.294143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1090/2157] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:43.294340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b974b-014d-4621-8fb4-5944d075a316 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:43.294697 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b974b-014d-4621-8fb4-5944d075a316 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:43.296645 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e2f8f-9e64-4122-8df9-b7a12c435f68 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:43.298682 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7c0b974b-014d-4621-8fb4-5944d075a316 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:43.299807 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-045e2f8f-9e64-4122-8df9-b7a12c435f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:43.323658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e2f8f-9e64-4122-8df9-b7a12c435f68 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:43.334415 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7cf4d-6f47-4ca1-bcd5-30644bdd95df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:43.335531 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7cf4d-6f47-4ca1-bcd5-30644bdd95df admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:43.336324 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7cf4d-6f47-4ca1-bcd5-30644bdd95df admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:43.337362 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7cf4d-6f47-4ca1-bcd5-30644bdd95df admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:43.338765 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f7cf4d-6f47-4ca1-bcd5-30644bdd95df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:43.340616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f7cf4d-6f47-4ca1-bcd5-30644bdd95df admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:43.351835 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1091/2158] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:43.355877 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64922072-f5ac-4b14-98d1-7a6db44c7e95 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:43.356165 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64922072-f5ac-4b14-98d1-7a6db44c7e95 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:43.356634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64922072-f5ac-4b14-98d1-7a6db44c7e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:43.387391 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-64922072-f5ac-4b14-98d1-7a6db44c7e95 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:43.388654 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-64922072-f5ac-4b14-98d1-7a6db44c7e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:43.391669 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b974b-014d-4621-8fb4-5944d075a316 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:43.393413 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b974b-014d-4621-8fb4-5944d075a316 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:43.397992 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1092/2159] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:43.399083 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e019b2a-7aa0-48fc-aba0-741306a88f03 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:43.399372 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e019b2a-7aa0-48fc-aba0-741306a88f03 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:43.399640 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e019b2a-7aa0-48fc-aba0-741306a88f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:43.413400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1068/2160] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:43 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:43.414694 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b974b-014d-4621-8fb4-5944d075a316 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:43.423698 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-0e019b2a-7aa0-48fc-aba0-741306a88f03 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:43.425806 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-0e019b2a-7aa0-48fc-aba0-741306a88f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:43.431468 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b974b-014d-4621-8fb4-5944d075a316 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:43.438926 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c3cd79-381a-4642-8fef-115a39a92a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:43.439935 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3cd79-381a-4642-8fef-115a39a92a2d admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:43.440363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3cd79-381a-4642-8fef-115a39a92a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:43.440779 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3cd79-381a-4642-8fef-115a39a92a2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:43.444652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1093/2161] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:43.444916 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c3cd79-381a-4642-8fef-115a39a92a2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:43.445017 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2539d2-f030-43be-a659-b2c93b63c955 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:43.445017 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2539d2-f030-43be-a659-b2c93b63c955 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:43.445017 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2539d2-f030-43be-a659-b2c93b63c955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:43.445603 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c3cd79-381a-4642-8fef-115a39a92a2d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:43.470677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1069/2162] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:43.472676 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c3cd79-381a-4642-8fef-115a39a92a2d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:43.483789 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-7e2539d2-f030-43be-a659-b2c93b63c955 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:43.485771 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-7e2539d2-f030-43be-a659-b2c93b63c955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:43.494353 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c3cd79-381a-4642-8fef-115a39a92a2d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:43.495875 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1094/2163] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:43.496895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2539d2-f030-43be-a659-b2c93b63c955 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:43.504138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455ec18-6419-4398-a83e-1d2874834b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:43.505539 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455ec18-6419-4398-a83e-1d2874834b5d admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:43.506131 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455ec18-6419-4398-a83e-1d2874834b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:43.506646 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455ec18-6419-4398-a83e-1d2874834b5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:43.508581 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455ec18-6419-4398-a83e-1d2874834b5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:13:43.509153 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455ec18-6419-4398-a83e-1d2874834b5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:43.511721 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2539d2-f030-43be-a659-b2c93b63c955 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:43.513088 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455ec18-6419-4398-a83e-1d2874834b5d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:43.517950 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dce4ad-382d-4cfb-8494-fe80c6b7f723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:43.519031 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dce4ad-382d-4cfb-8494-fe80c6b7f723 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:43.519259 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dce4ad-382d-4cfb-8494-fe80c6b7f723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:43.519566 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dce4ad-382d-4cfb-8494-fe80c6b7f723 admin admin] PATH_INFO: `/v3/projects/d0e9fe710b3243caa60ae2fc4418e0aa/users/1ded653401aa4e8592bb69490bd218ca/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:43.538474 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:43.543430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1070/2164] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:13:43.545546 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dce4ad-382d-4cfb-8494-fe80c6b7f723 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0e9fe710b3243caa60ae2fc4418e0aa, user_id=1ded653401aa4e8592bb69490bd218ca, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:43.546195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455ec18-6419-4398-a83e-1d2874834b5d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:43.553507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dce4ad-382d-4cfb-8494-fe80c6b7f723 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:43.570301 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455ec18-6419-4398-a83e-1d2874834b5d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:43.580463 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1095/2165] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:43 2022] PUT /identity/v3/projects/d0e9fe710b3243caa60ae2fc4418e0aa/users/1ded653401aa4e8592bb69490bd218ca/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:43.581196 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dce4ad-382d-4cfb-8494-fe80c6b7f723 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:43.590177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:43.695843 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995aa30-38e1-491b-b024-7371caf6afa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:43.698079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995aa30-38e1-491b-b024-7371caf6afa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:43.698948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995aa30-38e1-491b-b024-7371caf6afa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:43.699896 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995aa30-38e1-491b-b024-7371caf6afa3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:43.701336 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a995aa30-38e1-491b-b024-7371caf6afa3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:13:43.701818 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a995aa30-38e1-491b-b024-7371caf6afa3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:43.704332 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a995aa30-38e1-491b-b024-7371caf6afa3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:43.710408 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dce4ad-382d-4cfb-8494-fe80c6b7f723 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:43.718731 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4320f78a-3181-4617-9c9c-26bca1198659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:43.719796 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320f78a-3181-4617-9c9c-26bca1198659 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:43.720293 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320f78a-3181-4617-9c9c-26bca1198659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:43.721123 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320f78a-3181-4617-9c9c-26bca1198659 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:43.722032 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4320f78a-3181-4617-9c9c-26bca1198659 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:43.724224 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4320f78a-3181-4617-9c9c-26bca1198659 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:43.735050 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1071/2166] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:13:43.737780 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d41a3-8a3d-43cd-8b23-774b43d99658 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:43.738480 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d41a3-8a3d-43cd-8b23-774b43d99658 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:43.739081 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d41a3-8a3d-43cd-8b23-774b43d99658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:43.768122 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1096/2167] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:43 2022] POST /identity/v3/projects => generated 373 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:43.768122 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4320f78a-3181-4617-9c9c-26bca1198659 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:43.796702 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-030d41a3-8a3d-43cd-8b23-774b43d99658 None None] MFA Rules not processed for user `1ded653401aa4e8592bb69490bd218ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:43.800276 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-030d41a3-8a3d-43cd-8b23-774b43d99658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:43.903243 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4320f78a-3181-4617-9c9c-26bca1198659 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:43.912185 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5ad80-defb-41b5-864a-46d72096a2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:43.913009 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5ad80-defb-41b5-864a-46d72096a2c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:43.913558 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5ad80-defb-41b5-864a-46d72096a2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:43.914606 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5ad80-defb-41b5-864a-46d72096a2c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:43.915629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb5ad80-defb-41b5-864a-46d72096a2c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:43.917006 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1072/2168] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:43.918939 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d41a3-8a3d-43cd-8b23-774b43d99658 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:43.920822 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb5ad80-defb-41b5-864a-46d72096a2c2 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:43.940003 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d41a3-8a3d-43cd-8b23-774b43d99658 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:43.949011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1311ea2f-3f55-4243-bb86-98dd57ed6dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:43.950535 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311ea2f-3f55-4243-bb86-98dd57ed6dc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:43.951123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311ea2f-3f55-4243-bb86-98dd57ed6dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:43.951587 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311ea2f-3f55-4243-bb86-98dd57ed6dc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:43.952481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1311ea2f-3f55-4243-bb86-98dd57ed6dc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:43.954705 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1311ea2f-3f55-4243-bb86-98dd57ed6dc5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:43.973665 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1097/2169] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:43 2022] POST /identity/v3/projects => generated 373 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:43.975675 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5ad80-defb-41b5-864a-46d72096a2c2 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:43.980308 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-5fb5ad80-defb-41b5-864a-46d72096a2c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:43.983569 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1073/2170] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:43 2022] POST /identity/v3/users => generated 287 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:43.984948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1311ea2f-3f55-4243-bb86-98dd57ed6dc5 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:44.002975 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1311ea2f-3f55-4243-bb86-98dd57ed6dc5 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:44.083755 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5ad80-defb-41b5-864a-46d72096a2c2 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:44.091445 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a8880-4846-410b-8878-a54cd22d2bde service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f11beca9c5148eea4cc45f8dc89dbda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:44.092388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a8880-4846-410b-8878-a54cd22d2bde service glance] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:44.092822 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a8880-4846-410b-8878-a54cd22d2bde service glance] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:44.093325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a8880-4846-410b-8878-a54cd22d2bde service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:44.117174 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec0331e-5eed-4715-b68e-3fd91c3bb911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:44.118859 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0331e-5eed-4715-b68e-3fd91c3bb911 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:44.119082 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0331e-5eed-4715-b68e-3fd91c3bb911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:44.120069 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0331e-5eed-4715-b68e-3fd91c3bb911 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:44.121576 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec0331e-5eed-4715-b68e-3fd91c3bb911 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:44.126612 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec0331e-5eed-4715-b68e-3fd91c3bb911 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:44.127070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:44.187457 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1074/2171] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:43 2022] POST /identity/v3/users => generated 287 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:44.189400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec0331e-5eed-4715-b68e-3fd91c3bb911 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:44.213974 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec0331e-5eed-4715-b68e-3fd91c3bb911 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:44.238684 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d53da9-b6c7-4914-9fc3-dcb464a2fe68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:44.240212 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d53da9-b6c7-4914-9fc3-dcb464a2fe68 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:44.240757 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d53da9-b6c7-4914-9fc3-dcb464a2fe68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:44.241141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d53da9-b6c7-4914-9fc3-dcb464a2fe68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:44.242516 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d53da9-b6c7-4914-9fc3-dcb464a2fe68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:44.250243 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d53da9-b6c7-4914-9fc3-dcb464a2fe68 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:44.252898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:44.284282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1075/2172] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:44.285428 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a8880-4846-410b-8878-a54cd22d2bde service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:44.286178 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d53da9-b6c7-4914-9fc3-dcb464a2fe68 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:44.287831 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a8880-4846-410b-8878-a54cd22d2bde service glance] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:44.307022 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d53da9-b6c7-4914-9fc3-dcb464a2fe68 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:44.316781 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb135d3-46ec-4198-9cd4-72a901936550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:44.317695 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb135d3-46ec-4198-9cd4-72a901936550 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:44.318232 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1098/2173] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:44.320369 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a8880-4846-410b-8878-a54cd22d2bde service glance] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:44.331467 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb135d3-46ec-4198-9cd4-72a901936550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:44.331937 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb135d3-46ec-4198-9cd4-72a901936550 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:44.333259 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb135d3-46ec-4198-9cd4-72a901936550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:44.350021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a8880-4846-410b-8878-a54cd22d2bde service glance] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:44.350372 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb135d3-46ec-4198-9cd4-72a901936550 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:44.359882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b0d01-56ce-447b-848e-98029020abc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:44.361431 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b0d01-56ce-447b-848e-98029020abc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:44.362079 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b0d01-56ce-447b-848e-98029020abc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:44.362557 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b0d01-56ce-447b-848e-98029020abc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:44.364115 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86b0d01-56ce-447b-848e-98029020abc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:44.368139 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86b0d01-56ce-447b-848e-98029020abc7 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:44.396850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1076/2174] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:44.398503 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb135d3-46ec-4198-9cd4-72a901936550 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:44.402756 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1099/2175] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:44.404291 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b0d01-56ce-447b-848e-98029020abc7 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:44.419143 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb135d3-46ec-4198-9cd4-72a901936550 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:44.432434 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b0d01-56ce-447b-848e-98029020abc7 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:44.447923 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b12c3-6992-4a54-94b3-a32693c9e7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:44.447923 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b12c3-6992-4a54-94b3-a32693c9e7a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:44.447923 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b12c3-6992-4a54-94b3-a32693c9e7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:44.447923 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b12c3-6992-4a54-94b3-a32693c9e7a8 admin admin] PATH_INFO: `/v3/projects/10de7bebe555463c93b97f6ffc4d6dd2/users/87c14a4443724bf58d04d19103d50aff/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:44.452335 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4255a-9a5c-4020-9c18-40d711a481c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:44.454813 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4255a-9a5c-4020-9c18-40d711a481c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:44.455396 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4255a-9a5c-4020-9c18-40d711a481c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:44.455863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4255a-9a5c-4020-9c18-40d711a481c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:44.456802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a4255a-9a5c-4020-9c18-40d711a481c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:44.458912 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a4255a-9a5c-4020-9c18-40d711a481c7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:44.473287 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1077/2176] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:44.478554 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4255a-9a5c-4020-9c18-40d711a481c7 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:44.488873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b12c3-6992-4a54-94b3-a32693c9e7a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10de7bebe555463c93b97f6ffc4d6dd2, user_id=87c14a4443724bf58d04d19103d50aff, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:44.497654 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b12c3-6992-4a54-94b3-a32693c9e7a8 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:44.498976 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4255a-9a5c-4020-9c18-40d711a481c7 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:44.504794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b97b077-8358-475c-a582-ec18ded77db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:44.506169 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97b077-8358-475c-a582-ec18ded77db9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:44.506620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97b077-8358-475c-a582-ec18ded77db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:44.507355 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97b077-8358-475c-a582-ec18ded77db9 admin admin] PATH_INFO: `/v3/projects/7b4ba60390dd4354b869bb227ef030b6/users/246ac84c69894c08b870c4579cabd042/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:44.532230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1100/2177] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:44 2022] PUT /identity/v3/projects/10de7bebe555463c93b97f6ffc4d6dd2/users/87c14a4443724bf58d04d19103d50aff/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:44.541999 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b97b077-8358-475c-a582-ec18ded77db9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b4ba60390dd4354b869bb227ef030b6, user_id=246ac84c69894c08b870c4579cabd042, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:44.548496 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2bb08-3847-48e7-97ff-766e36588c4f None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:44.548907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2bb08-3847-48e7-97ff-766e36588c4f None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:44.549223 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2bb08-3847-48e7-97ff-766e36588c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:44.555555 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b97b077-8358-475c-a582-ec18ded77db9 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:44.585111 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1078/2178] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:44 2022] PUT /identity/v3/projects/7b4ba60390dd4354b869bb227ef030b6/users/246ac84c69894c08b870c4579cabd042/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:44.591787 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad579dc-6afc-48a4-8264-b8a028e52290 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:44.594669 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad579dc-6afc-48a4-8264-b8a028e52290 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:44.595209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad579dc-6afc-48a4-8264-b8a028e52290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:44.598645 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-7aa2bb08-3847-48e7-97ff-766e36588c4f None None] MFA Rules not processed for user `87c14a4443724bf58d04d19103d50aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:44.600445 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-7aa2bb08-3847-48e7-97ff-766e36588c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:44.628145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0ad579dc-6afc-48a4-8264-b8a028e52290 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:44.629612 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0ad579dc-6afc-48a4-8264-b8a028e52290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:44.739583 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1101/2179] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:44.755361 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2bb08-3847-48e7-97ff-766e36588c4f None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:44.757192 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-7aa2bb08-3847-48e7-97ff-766e36588c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:44.790011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1079/2180] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:13:44.804791 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad579dc-6afc-48a4-8264-b8a028e52290 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:44.808565 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0ad579dc-6afc-48a4-8264-b8a028e52290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:44.901724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad579dc-6afc-48a4-8264-b8a028e52290 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:44.904838 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2bb08-3847-48e7-97ff-766e36588c4f None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:44.905699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf72b52-b179-4864-8953-eb6996bff18d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:44.906409 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf72b52-b179-4864-8953-eb6996bff18d service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:44.906687 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf72b52-b179-4864-8953-eb6996bff18d service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:44.906967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf72b52-b179-4864-8953-eb6996bff18d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:44.914312 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0bd1f-e797-4db7-8fb0-238d7b5c4775 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:44.915284 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0bd1f-e797-4db7-8fb0-238d7b5c4775 service nova] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:44.921071 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0bd1f-e797-4db7-8fb0-238d7b5c4775 service nova] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:44.921071 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0bd1f-e797-4db7-8fb0-238d7b5c4775 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:44.923388 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf72b52-b179-4864-8953-eb6996bff18d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:44.925022 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf72b52-b179-4864-8953-eb6996bff18d service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:44.931022 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:44.935568 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e0bd1f-e797-4db7-8fb0-238d7b5c4775 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:44.938188 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e0bd1f-e797-4db7-8fb0-238d7b5c4775 service nova] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:44.940301 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1080/2181] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:44.980369 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1102/2182] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:46.204187 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd152e-d27d-42be-ba01-f86ac2c2fbc6 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:46.205803 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd152e-d27d-42be-ba01-f86ac2c2fbc6 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:46.206801 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd152e-d27d-42be-ba01-f86ac2c2fbc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:46.283840 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f6cd152e-d27d-42be-ba01-f86ac2c2fbc6 None None] MFA Rules not processed for user `246ac84c69894c08b870c4579cabd042`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:46.286323 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f6cd152e-d27d-42be-ba01-f86ac2c2fbc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:46.387215 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:46.433159 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1081/2183] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:46.444930 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0bd1f-e797-4db7-8fb0-238d7b5c4775 service nova] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:46.549956 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0bd1f-e797-4db7-8fb0-238d7b5c4775 service nova] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:46.554534 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13961a-f3b6-4959-bcb3-e8f9fe5e4137 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:46.555230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13961a-f3b6-4959-bcb3-e8f9fe5e4137 service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:46.555519 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13961a-f3b6-4959-bcb3-e8f9fe5e4137 service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:46.555802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13961a-f3b6-4959-bcb3-e8f9fe5e4137 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:46.566806 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13961a-f3b6-4959-bcb3-e8f9fe5e4137 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:46.568860 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13961a-f3b6-4959-bcb3-e8f9fe5e4137 service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:46.581125 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1103/2184] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:50.344148 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13647e-e583-4a07-a5ca-f1c438628205 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:50.344933 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13647e-e583-4a07-a5ca-f1c438628205 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:50.345077 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13647e-e583-4a07-a5ca-f1c438628205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:50.371129 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-5e13647e-e583-4a07-a5ca-f1c438628205 None None] MFA Rules not processed for user `246ac84c69894c08b870c4579cabd042`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:50.372595 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5e13647e-e583-4a07-a5ca-f1c438628205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:50.380582 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1082/2185] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:50.613087 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13961a-f3b6-4959-bcb3-e8f9fe5e4137 service cinder] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:50.630610 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13961a-f3b6-4959-bcb3-e8f9fe5e4137 service cinder] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:50.638882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d7cd4-df60-4f88-8926-c575016b50f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:50.640376 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d7cd4-df60-4f88-8926-c575016b50f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:50.640853 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d7cd4-df60-4f88-8926-c575016b50f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:50.641288 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d7cd4-df60-4f88-8926-c575016b50f1 admin admin] PATH_INFO: `/v3/users/87c14a4443724bf58d04d19103d50aff` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:50.644009 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d7cd4-df60-4f88-8926-c575016b50f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87c14a4443724bf58d04d19103d50aff)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:50.652895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d7cd4-df60-4f88-8926-c575016b50f1 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:50.732277 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-413d7cd4-df60-4f88-8926-c575016b50f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87c14a4443724bf58d04d19103d50aff'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:50.748495 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-413d7cd4-df60-4f88-8926-c575016b50f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87c14a4443724bf58d04d19103d50aff'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:50.778865 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-413d7cd4-df60-4f88-8926-c575016b50f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87c14a4443724bf58d04d19103d50aff'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:50.780538 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-413d7cd4-df60-4f88-8926-c575016b50f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87c14a4443724bf58d04d19103d50aff'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:50.802258 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-413d7cd4-df60-4f88-8926-c575016b50f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87c14a4443724bf58d04d19103d50aff'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:50.817409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1104/2186] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:50 2022] DELETE /identity/v3/users/87c14a4443724bf58d04d19103d50aff => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:50.825623 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc18297-0750-488b-808d-66e68e2d95f1 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:50.826228 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc18297-0750-488b-808d-66e68e2d95f1 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:50.826642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc18297-0750-488b-808d-66e68e2d95f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:50.858375 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ffc18297-0750-488b-808d-66e68e2d95f1 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:50.860422 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ffc18297-0750-488b-808d-66e68e2d95f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:50.996854 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1083/2187] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:51.014792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d7cd4-df60-4f88-8926-c575016b50f1 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:51.017131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-413d7cd4-df60-4f88-8926-c575016b50f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:51.142374 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d7cd4-df60-4f88-8926-c575016b50f1 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:51.149577 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc6f81-8f2f-4c64-a90c-4fc0d5b5ae26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:51.151724 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc6f81-8f2f-4c64-a90c-4fc0d5b5ae26 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:51.152349 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc6f81-8f2f-4c64-a90c-4fc0d5b5ae26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:51.152694 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc6f81-8f2f-4c64-a90c-4fc0d5b5ae26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:51.164977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc6f81-8f2f-4c64-a90c-4fc0d5b5ae26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:51.167761 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc6f81-8f2f-4c64-a90c-4fc0d5b5ae26 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:51.183867 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1105/2188] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:51.640773 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc18297-0750-488b-808d-66e68e2d95f1 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:51.643683 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ffc18297-0750-488b-808d-66e68e2d95f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:51.686334 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc18297-0750-488b-808d-66e68e2d95f1 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:51.708893 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8cd65-3fc4-40c5-a21c-cfee0bb2896e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:51.710747 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8cd65-3fc4-40c5-a21c-cfee0bb2896e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:51.711379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8cd65-3fc4-40c5-a21c-cfee0bb2896e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:51.712406 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8cd65-3fc4-40c5-a21c-cfee0bb2896e admin admin] PATH_INFO: `/v3/projects/10de7bebe555463c93b97f6ffc4d6dd2` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:51.715124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8cd65-3fc4-40c5-a21c-cfee0bb2896e admin admin] RBAC: Authorizing `identity:delete_project(project_id=10de7bebe555463c93b97f6ffc4d6dd2)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:51.717671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8cd65-3fc4-40c5-a21c-cfee0bb2896e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:51.803811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-64c8cd65-3fc4-40c5-a21c-cfee0bb2896e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10de7bebe555463c93b97f6ffc4d6dd2', 'request_id': 'req-64c8cd65-3fc4-40c5-a21c-cfee0bb2896e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:51.813599 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-64c8cd65-3fc4-40c5-a21c-cfee0bb2896e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10de7bebe555463c93b97f6ffc4d6dd2', 'request_id': 'req-64c8cd65-3fc4-40c5-a21c-cfee0bb2896e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:51.835294 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-64c8cd65-3fc4-40c5-a21c-cfee0bb2896e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10de7bebe555463c93b97f6ffc4d6dd2', 'request_id': 'req-64c8cd65-3fc4-40c5-a21c-cfee0bb2896e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:51.852076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-64c8cd65-3fc4-40c5-a21c-cfee0bb2896e admin admin] The token cache is being invalidate because project 10de7bebe555463c93b97f6ffc4d6dd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:51.852388 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-64c8cd65-3fc4-40c5-a21c-cfee0bb2896e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:51.853106 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-64c8cd65-3fc4-40c5-a21c-cfee0bb2896e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:51.858026 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1084/2189] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:51 2022] DELETE /identity/v3/projects/10de7bebe555463c93b97f6ffc4d6dd2 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:51.860630 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278c43b-540e-4954-a3b4-faafccdb791d None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:51.861101 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278c43b-540e-4954-a3b4-faafccdb791d None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:51.861191 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278c43b-540e-4954-a3b4-faafccdb791d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:51.888171 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-2278c43b-540e-4954-a3b4-faafccdb791d None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:51.889513 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-2278c43b-540e-4954-a3b4-faafccdb791d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:51.900419 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:51.988727 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1106/2190] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:51.998513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875be73-73e2-46ad-96fe-6717094b3c24 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:51.998796 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875be73-73e2-46ad-96fe-6717094b3c24 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:51.999073 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875be73-73e2-46ad-96fe-6717094b3c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:52.023419 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-6875be73-73e2-46ad-96fe-6717094b3c24 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:52.025080 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6875be73-73e2-46ad-96fe-6717094b3c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:52.038115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1085/2191] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:13:52.045513 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278c43b-540e-4954-a3b4-faafccdb791d None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:52.060749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278c43b-540e-4954-a3b4-faafccdb791d None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:52.066596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ed39a-a247-480d-9b90-b4e749ea84f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:52.067300 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ed39a-a247-480d-9b90-b4e749ea84f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:52.067578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ed39a-a247-480d-9b90-b4e749ea84f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:52.067850 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ed39a-a247-480d-9b90-b4e749ea84f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:52.068595 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1ed39a-a247-480d-9b90-b4e749ea84f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:13:52.068900 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1ed39a-a247-480d-9b90-b4e749ea84f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:52.070625 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1ed39a-a247-480d-9b90-b4e749ea84f8 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:52.094642 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1107/2192] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:13:52.096192 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6875be73-73e2-46ad-96fe-6717094b3c24 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:52.109876 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6875be73-73e2-46ad-96fe-6717094b3c24 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:52.115253 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8a516-31e2-4225-8dd8-d28b1983ba11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:52.115956 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8a516-31e2-4225-8dd8-d28b1983ba11 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:52.116235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8a516-31e2-4225-8dd8-d28b1983ba11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:52.116505 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8a516-31e2-4225-8dd8-d28b1983ba11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:52.117215 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8a516-31e2-4225-8dd8-d28b1983ba11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:52.119725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8a516-31e2-4225-8dd8-d28b1983ba11 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:52.157158 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1086/2193] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:52 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:52.162855 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ed39a-a247-480d-9b90-b4e749ea84f8 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:52.174466 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ed39a-a247-480d-9b90-b4e749ea84f8 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:52.284609 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca46b2-6b99-4356-a06b-2a6fe9549d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:52.285290 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca46b2-6b99-4356-a06b-2a6fe9549d84 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:52.285587 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca46b2-6b99-4356-a06b-2a6fe9549d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:52.285862 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca46b2-6b99-4356-a06b-2a6fe9549d84 admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:52.286755 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca46b2-6b99-4356-a06b-2a6fe9549d84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:52.288943 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca46b2-6b99-4356-a06b-2a6fe9549d84 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:52.321602 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1108/2194] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:52 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:52.343886 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8a516-31e2-4225-8dd8-d28b1983ba11 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:52.358168 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8a516-31e2-4225-8dd8-d28b1983ba11 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:52.363682 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9356a9-b1ab-4255-80ce-00e58b3a7960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:52.364383 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9356a9-b1ab-4255-80ce-00e58b3a7960 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:52.364664 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9356a9-b1ab-4255-80ce-00e58b3a7960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:52.364940 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9356a9-b1ab-4255-80ce-00e58b3a7960 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:52.365682 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9356a9-b1ab-4255-80ce-00e58b3a7960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:52.367379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9356a9-b1ab-4255-80ce-00e58b3a7960 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:52.379135 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1087/2195] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:52.383955 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca46b2-6b99-4356-a06b-2a6fe9549d84 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:52.395062 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca46b2-6b99-4356-a06b-2a6fe9549d84 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:52.402616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc5a0b-468a-47dc-a19d-07d78fe8d628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:52.403331 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc5a0b-468a-47dc-a19d-07d78fe8d628 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:52.403616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc5a0b-468a-47dc-a19d-07d78fe8d628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:52.403922 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc5a0b-468a-47dc-a19d-07d78fe8d628 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:52.404693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afc5a0b-468a-47dc-a19d-07d78fe8d628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:52.406682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afc5a0b-468a-47dc-a19d-07d78fe8d628 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:52.419633 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1109/2196] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:52.426647 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9356a9-b1ab-4255-80ce-00e58b3a7960 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:52.452788 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9356a9-b1ab-4255-80ce-00e58b3a7960 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:52.460059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243580e-a4bc-4396-bd09-fb6313b97aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:52.460986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243580e-a4bc-4396-bd09-fb6313b97aa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:52.461337 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243580e-a4bc-4396-bd09-fb6313b97aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:52.461573 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243580e-a4bc-4396-bd09-fb6313b97aa7 admin admin] PATH_INFO: `/v3/projects/b3c03ed4e63b41958094fb3315a04d1e/users/5573930262ff4d7988e0124b27fbf7e8/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:52.483996 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d243580e-a4bc-4396-bd09-fb6313b97aa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3c03ed4e63b41958094fb3315a04d1e, user_id=5573930262ff4d7988e0124b27fbf7e8, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:52.486933 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d243580e-a4bc-4396-bd09-fb6313b97aa7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:52.492128 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:52.513191 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1088/2197] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:52 2022] PUT /identity/v3/projects/b3c03ed4e63b41958094fb3315a04d1e/users/5573930262ff4d7988e0124b27fbf7e8/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:52.533200 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638e05b-23e6-4d66-8f3b-f082c7e42b57 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:52.534007 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638e05b-23e6-4d66-8f3b-f082c7e42b57 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:52.534328 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638e05b-23e6-4d66-8f3b-f082c7e42b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:52.575134 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243580e-a4bc-4396-bd09-fb6313b97aa7 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:52.576551 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d243580e-a4bc-4396-bd09-fb6313b97aa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:52.591879 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-7638e05b-23e6-4d66-8f3b-f082c7e42b57 None None] MFA Rules not processed for user `5573930262ff4d7988e0124b27fbf7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:52.593307 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-7638e05b-23e6-4d66-8f3b-f082c7e42b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:52.700450 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243580e-a4bc-4396-bd09-fb6313b97aa7 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:52.707002 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bed82f-a2b0-4174-bc49-cb77968fbfcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:52.707915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bed82f-a2b0-4174-bc49-cb77968fbfcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:52.708205 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bed82f-a2b0-4174-bc49-cb77968fbfcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:52.708552 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bed82f-a2b0-4174-bc49-cb77968fbfcc admin admin] PATH_INFO: `/v3/users/e31cd745f6e04f1f998f57ab6bbe934f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:52.710294 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bed82f-a2b0-4174-bc49-cb77968fbfcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e31cd745f6e04f1f998f57ab6bbe934f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:52.711769 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1110/2198] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:52.712268 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bed82f-a2b0-4174-bc49-cb77968fbfcc admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:52.719151 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638e05b-23e6-4d66-8f3b-f082c7e42b57 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:52.737601 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638e05b-23e6-4d66-8f3b-f082c7e42b57 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:52.751531 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01e758-5062-468a-8f23-aa5904ca69e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:52.752232 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01e758-5062-468a-8f23-aa5904ca69e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:52.752526 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01e758-5062-468a-8f23-aa5904ca69e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:52.752839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01e758-5062-468a-8f23-aa5904ca69e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:52.753679 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01e758-5062-468a-8f23-aa5904ca69e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:52.755826 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01e758-5062-468a-8f23-aa5904ca69e4 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:52.769715 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:52.819448 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1111/2199] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:52 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:13:52.825313 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-23bed82f-a2b0-4174-bc49-cb77968fbfcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e31cd745f6e04f1f998f57ab6bbe934f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:52.826159 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01e758-5062-468a-8f23-aa5904ca69e4 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:52.857964 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-23bed82f-a2b0-4174-bc49-cb77968fbfcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e31cd745f6e04f1f998f57ab6bbe934f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:52.868582 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-23bed82f-a2b0-4174-bc49-cb77968fbfcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e31cd745f6e04f1f998f57ab6bbe934f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:52.878602 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-23bed82f-a2b0-4174-bc49-cb77968fbfcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e31cd745f6e04f1f998f57ab6bbe934f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:52.879842 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-23bed82f-a2b0-4174-bc49-cb77968fbfcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e31cd745f6e04f1f998f57ab6bbe934f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:52.892932 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1089/2200] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:52 2022] DELETE /identity/v3/users/e31cd745f6e04f1f998f57ab6bbe934f => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:52.899254 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26add5-6cb0-4ebd-9c70-e5f0ae03f6c3 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:52.899930 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26add5-6cb0-4ebd-9c70-e5f0ae03f6c3 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:52.900574 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26add5-6cb0-4ebd-9c70-e5f0ae03f6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:52.929371 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-bd26add5-6cb0-4ebd-9c70-e5f0ae03f6c3 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:52.931454 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bd26add5-6cb0-4ebd-9c70-e5f0ae03f6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:52.932630 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01e758-5062-468a-8f23-aa5904ca69e4 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:53.026923 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1090/2201] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:13:53.027145 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b935051-e646-4fda-9133-d790a05a8a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:53.027145 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b935051-e646-4fda-9133-d790a05a8a0d admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:53.027145 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b935051-e646-4fda-9133-d790a05a8a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:53.027145 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b935051-e646-4fda-9133-d790a05a8a0d admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:53.027482 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b935051-e646-4fda-9133-d790a05a8a0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:53.028860 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b935051-e646-4fda-9133-d790a05a8a0d admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:53.035514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26add5-6cb0-4ebd-9c70-e5f0ae03f6c3 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:53.037836 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bd26add5-6cb0-4ebd-9c70-e5f0ae03f6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:53.074621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1112/2202] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:52 2022] POST /identity/v3/users => generated 293 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:53.080129 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b935051-e646-4fda-9133-d790a05a8a0d admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:53.081147 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:53.082600 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-5b935051-e646-4fda-9133-d790a05a8a0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:53.104007 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b935051-e646-4fda-9133-d790a05a8a0d admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:53.110883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d7336-dbac-4562-926e-4b055224d7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:53.111590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d7336-dbac-4562-926e-4b055224d7d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:53.111868 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d7336-dbac-4562-926e-4b055224d7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:53.112148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d7336-dbac-4562-926e-4b055224d7d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:53.112901 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d7336-dbac-4562-926e-4b055224d7d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:53.114593 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d7336-dbac-4562-926e-4b055224d7d0 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:53.144455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1113/2203] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:13:53.146185 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d7336-dbac-4562-926e-4b055224d7d0 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:53.160383 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d7336-dbac-4562-926e-4b055224d7d0 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:53.166572 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1bd77-c2a4-4c2f-8a1e-f6d48c2d7c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:53.166887 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26add5-6cb0-4ebd-9c70-e5f0ae03f6c3 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:53.167555 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1bd77-c2a4-4c2f-8a1e-f6d48c2d7c53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:53.167866 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1bd77-c2a4-4c2f-8a1e-f6d48c2d7c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:53.168182 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1bd77-c2a4-4c2f-8a1e-f6d48c2d7c53 admin admin] PATH_INFO: `/v3/projects/cbf8ddcc1b344751a1cde419f7394d46/users/017b42a077174663939a26ddee024222/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:53.171150 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb8912-95b4-45dd-a004-06ebae387362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:53.171817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb8912-95b4-45dd-a004-06ebae387362 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:53.172099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb8912-95b4-45dd-a004-06ebae387362 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:53.172375 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb8912-95b4-45dd-a004-06ebae387362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:53.185274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb8912-95b4-45dd-a004-06ebae387362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:53.186084 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb8912-95b4-45dd-a004-06ebae387362 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:53.186780 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1bd77-c2a4-4c2f-8a1e-f6d48c2d7c53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbf8ddcc1b344751a1cde419f7394d46, user_id=017b42a077174663939a26ddee024222, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:53.189909 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1bd77-c2a4-4c2f-8a1e-f6d48c2d7c53 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:53.198401 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1091/2204] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:13:53.209206 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1114/2205] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:53 2022] PUT /identity/v3/projects/cbf8ddcc1b344751a1cde419f7394d46/users/017b42a077174663939a26ddee024222/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:53.214359 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb8912-95b4-45dd-a004-06ebae387362 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:53.224650 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb8912-95b4-45dd-a004-06ebae387362 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:53.330233 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367c52e-e6eb-441f-a512-f47e187d6567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:53.331385 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367c52e-e6eb-441f-a512-f47e187d6567 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:53.331863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367c52e-e6eb-441f-a512-f47e187d6567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:53.332380 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367c52e-e6eb-441f-a512-f47e187d6567 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:53.333484 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7367c52e-e6eb-441f-a512-f47e187d6567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:53.336302 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7367c52e-e6eb-441f-a512-f47e187d6567 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:53.357744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1092/2206] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:13:53.363194 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1bd77-c2a4-4c2f-8a1e-f6d48c2d7c53 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:53.388528 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1bd77-c2a4-4c2f-8a1e-f6d48c2d7c53 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:53.396529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14336b45-2fbf-4db5-acbe-284a0af23056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:53.398008 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14336b45-2fbf-4db5-acbe-284a0af23056 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:53.398413 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14336b45-2fbf-4db5-acbe-284a0af23056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:53.398822 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14336b45-2fbf-4db5-acbe-284a0af23056 admin admin] PATH_INFO: `/v3/projects/cbf8ddcc1b344751a1cde419f7394d46/users/017b42a077174663939a26ddee024222/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:53.402278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14336b45-2fbf-4db5-acbe-284a0af23056 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbf8ddcc1b344751a1cde419f7394d46, user_id=017b42a077174663939a26ddee024222, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:53.406390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14336b45-2fbf-4db5-acbe-284a0af23056 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:53.430188 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1115/2207] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:53 2022] PUT /identity/v3/projects/cbf8ddcc1b344751a1cde419f7394d46/users/017b42a077174663939a26ddee024222/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:53.450647 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf5992-bec4-4304-8f17-495776fd4fb0 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:53.451002 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf5992-bec4-4304-8f17-495776fd4fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:53.451311 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf5992-bec4-4304-8f17-495776fd4fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:53.497302 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-64cf5992-bec4-4304-8f17-495776fd4fb0 None None] MFA Rules not processed for user `017b42a077174663939a26ddee024222`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:53.499511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-64cf5992-bec4-4304-8f17-495776fd4fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:53.594819 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14336b45-2fbf-4db5-acbe-284a0af23056 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:53.599962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-14336b45-2fbf-4db5-acbe-284a0af23056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:53.674085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1093/2208] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:13:53 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:13:53.688931 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:53.692339 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf5992-bec4-4304-8f17-495776fd4fb0 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:53.694569 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-64cf5992-bec4-4304-8f17-495776fd4fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:53.761850 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14336b45-2fbf-4db5-acbe-284a0af23056 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:53.767731 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2421f5-f536-4e59-8eda-a1f596933961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:53.768670 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2421f5-f536-4e59-8eda-a1f596933961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:53.769138 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2421f5-f536-4e59-8eda-a1f596933961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:53.769609 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2421f5-f536-4e59-8eda-a1f596933961 admin admin] PATH_INFO: `/v3/projects/a9843ebd86dc49608e83ce0ad80c678e` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:53.771034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2421f5-f536-4e59-8eda-a1f596933961 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9843ebd86dc49608e83ce0ad80c678e)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:53.773440 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2421f5-f536-4e59-8eda-a1f596933961 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:53.797382 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:53.826995 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf5992-bec4-4304-8f17-495776fd4fb0 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:53.834630 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead9df1-0449-4a68-8adc-6b1f30cdedc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:53.837125 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead9df1-0449-4a68-8adc-6b1f30cdedc0 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:53.838666 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead9df1-0449-4a68-8adc-6b1f30cdedc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:53.840367 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead9df1-0449-4a68-8adc-6b1f30cdedc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:53.873404 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead9df1-0449-4a68-8adc-6b1f30cdedc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:53.877318 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead9df1-0449-4a68-8adc-6b1f30cdedc0 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:53.896164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-da2421f5-f536-4e59-8eda-a1f596933961 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9843ebd86dc49608e83ce0ad80c678e', 'request_id': 'req-da2421f5-f536-4e59-8eda-a1f596933961'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:53.897402 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1094/2209] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:53 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:53.930255 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-da2421f5-f536-4e59-8eda-a1f596933961 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9843ebd86dc49608e83ce0ad80c678e', 'request_id': 'req-da2421f5-f536-4e59-8eda-a1f596933961'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:53.943053 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-da2421f5-f536-4e59-8eda-a1f596933961 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9843ebd86dc49608e83ce0ad80c678e', 'request_id': 'req-da2421f5-f536-4e59-8eda-a1f596933961'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:53.965214 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-da2421f5-f536-4e59-8eda-a1f596933961 admin admin] The token cache is being invalidate because project a9843ebd86dc49608e83ce0ad80c678e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:53.965903 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-da2421f5-f536-4e59-8eda-a1f596933961 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:53.966857 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-da2421f5-f536-4e59-8eda-a1f596933961 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:53.969327 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1116/2210] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:53 2022] DELETE /identity/v3/projects/a9843ebd86dc49608e83ce0ad80c678e => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:54.015874 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead9df1-0449-4a68-8adc-6b1f30cdedc0 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:54.026846 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead9df1-0449-4a68-8adc-6b1f30cdedc0 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:54.115637 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99cebd-b2d9-4117-b5ab-9ad7dbae2036 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:54.116301 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99cebd-b2d9-4117-b5ab-9ad7dbae2036 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:54.116587 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99cebd-b2d9-4117-b5ab-9ad7dbae2036 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:54.117012 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99cebd-b2d9-4117-b5ab-9ad7dbae2036 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:54.119771 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bf99cebd-b2d9-4117-b5ab-9ad7dbae2036 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:54.237739 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99cebd-b2d9-4117-b5ab-9ad7dbae2036 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:54.239781 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99cebd-b2d9-4117-b5ab-9ad7dbae2036 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:54.251368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1095/2211] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:14:03.958965 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=35,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:04.248077 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=56 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:28.984642 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2421f5-f536-4e59-8eda-a1f596933961 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:28.987195 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-da2421f5-f536-4e59-8eda-a1f596933961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:29.268476 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2421f5-f536-4e59-8eda-a1f596933961 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:29.283439 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6007d30-8e34-4246-a12f-2cf4679eab0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:29.284807 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6007d30-8e34-4246-a12f-2cf4679eab0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:29.285418 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6007d30-8e34-4246-a12f-2cf4679eab0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:29.285984 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6007d30-8e34-4246-a12f-2cf4679eab0d admin admin] PATH_INFO: `/v3/users/1ded653401aa4e8592bb69490bd218ca` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:29.288303 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6007d30-8e34-4246-a12f-2cf4679eab0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ded653401aa4e8592bb69490bd218ca)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:29.295740 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6007d30-8e34-4246-a12f-2cf4679eab0d admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:29.386066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-a6007d30-8e34-4246-a12f-2cf4679eab0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ded653401aa4e8592bb69490bd218ca'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:29.396122 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-a6007d30-8e34-4246-a12f-2cf4679eab0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ded653401aa4e8592bb69490bd218ca'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:29.422733 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-a6007d30-8e34-4246-a12f-2cf4679eab0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ded653401aa4e8592bb69490bd218ca'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:29.424655 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-a6007d30-8e34-4246-a12f-2cf4679eab0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ded653401aa4e8592bb69490bd218ca'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:29.446776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-a6007d30-8e34-4246-a12f-2cf4679eab0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ded653401aa4e8592bb69490bd218ca'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:29.455351 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1117/2212] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:14:28 2022] DELETE /identity/v3/users/1ded653401aa4e8592bb69490bd218ca => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:14:29.462582 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4de88-966f-47b8-8e6f-886f2da8f2f7 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:29.463251 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4de88-966f-47b8-8e6f-886f2da8f2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:29.463757 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4de88-966f-47b8-8e6f-886f2da8f2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:29.519176 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-cdc4de88-966f-47b8-8e6f-886f2da8f2f7 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:29.520327 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-cdc4de88-966f-47b8-8e6f-886f2da8f2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:29.637770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1096/2213] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:14:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:29.658002 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6007d30-8e34-4246-a12f-2cf4679eab0d admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:29.661270 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-a6007d30-8e34-4246-a12f-2cf4679eab0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:29.718533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:29.872208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6007d30-8e34-4246-a12f-2cf4679eab0d admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:29.878946 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545eeaf-a65b-4ab3-9971-99015b78c921 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:29.880289 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545eeaf-a65b-4ab3-9971-99015b78c921 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:29.880979 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545eeaf-a65b-4ab3-9971-99015b78c921 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:29.881530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545eeaf-a65b-4ab3-9971-99015b78c921 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:29.900888 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545eeaf-a65b-4ab3-9971-99015b78c921 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:29.903447 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545eeaf-a65b-4ab3-9971-99015b78c921 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:29.920408 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1118/2214] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:14:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:14:30.325986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc4de88-966f-47b8-8e6f-886f2da8f2f7 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:30.328259 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-cdc4de88-966f-47b8-8e6f-886f2da8f2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:30.361008 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc4de88-966f-47b8-8e6f-886f2da8f2f7 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:30.367247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7acd61-29a8-4de7-a211-8ca9d245d837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:30.369133 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7acd61-29a8-4de7-a211-8ca9d245d837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:30.369718 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7acd61-29a8-4de7-a211-8ca9d245d837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:30.370016 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7acd61-29a8-4de7-a211-8ca9d245d837 admin admin] PATH_INFO: `/v3/projects/d0e9fe710b3243caa60ae2fc4418e0aa` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:30.371889 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7acd61-29a8-4de7-a211-8ca9d245d837 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0e9fe710b3243caa60ae2fc4418e0aa)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:30.374976 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7acd61-29a8-4de7-a211-8ca9d245d837 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:30.472538 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4b7acd61-29a8-4de7-a211-8ca9d245d837 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0e9fe710b3243caa60ae2fc4418e0aa', 'request_id': 'req-4b7acd61-29a8-4de7-a211-8ca9d245d837'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:30.482733 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4b7acd61-29a8-4de7-a211-8ca9d245d837 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0e9fe710b3243caa60ae2fc4418e0aa', 'request_id': 'req-4b7acd61-29a8-4de7-a211-8ca9d245d837'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:30.493825 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4b7acd61-29a8-4de7-a211-8ca9d245d837 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0e9fe710b3243caa60ae2fc4418e0aa', 'request_id': 'req-4b7acd61-29a8-4de7-a211-8ca9d245d837'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:30.513151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4b7acd61-29a8-4de7-a211-8ca9d245d837 admin admin] The token cache is being invalidate because project d0e9fe710b3243caa60ae2fc4418e0aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:14:30.513465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4b7acd61-29a8-4de7-a211-8ca9d245d837 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:30.514288 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4b7acd61-29a8-4de7-a211-8ca9d245d837 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:30.516196 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1097/2215] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:14:30 2022] DELETE /identity/v3/projects/d0e9fe710b3243caa60ae2fc4418e0aa => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:30.523373 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0161c12-1868-4cb1-82a0-023c79641f55 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:30.524382 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0161c12-1868-4cb1-82a0-023c79641f55 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:30.524579 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0161c12-1868-4cb1-82a0-023c79641f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:30.612874 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-b0161c12-1868-4cb1-82a0-023c79641f55 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:30.615225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-b0161c12-1868-4cb1-82a0-023c79641f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:30.718744 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1119/2216] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:14:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:14:30.739205 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843efa5b-095a-4341-b8d8-df963ea0846f None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:30.739679 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843efa5b-095a-4341-b8d8-df963ea0846f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:30.740461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843efa5b-095a-4341-b8d8-df963ea0846f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:30.759630 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0161c12-1868-4cb1-82a0-023c79641f55 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:30.762204 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-b0161c12-1868-4cb1-82a0-023c79641f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:30.772282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-843efa5b-095a-4341-b8d8-df963ea0846f None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:30.773966 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-843efa5b-095a-4341-b8d8-df963ea0846f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:30.789529 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1098/2217] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:14:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:30.796071 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843efa5b-095a-4341-b8d8-df963ea0846f None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:30.799375 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0161c12-1868-4cb1-82a0-023c79641f55 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:30.813535 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843efa5b-095a-4341-b8d8-df963ea0846f None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:30.814805 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba8f7b-0923-4300-aef9-9ae4f13fc1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:30.818218 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba8f7b-0923-4300-aef9-9ae4f13fc1d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:30.819120 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba8f7b-0923-4300-aef9-9ae4f13fc1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:30.820859 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba8f7b-0923-4300-aef9-9ae4f13fc1d8 admin admin] PATH_INFO: `/v3/users/7d5ec1ea62414eb5b76cb9135b0515c2` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:30.822677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998865d-1dd4-4cc3-b1b2-75bc125bd5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:30.823647 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998865d-1dd4-4cc3-b1b2-75bc125bd5a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:30.824090 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998865d-1dd4-4cc3-b1b2-75bc125bd5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:30.824534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998865d-1dd4-4cc3-b1b2-75bc125bd5a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:30.825320 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ba8f7b-0923-4300-aef9-9ae4f13fc1d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d5ec1ea62414eb5b76cb9135b0515c2)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:30.825990 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7998865d-1dd4-4cc3-b1b2-75bc125bd5a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:14:30.826337 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7998865d-1dd4-4cc3-b1b2-75bc125bd5a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:30.830083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7998865d-1dd4-4cc3-b1b2-75bc125bd5a8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:30.832416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ba8f7b-0923-4300-aef9-9ae4f13fc1d8 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:30.860517 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1099/2218] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:14:30.868980 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998865d-1dd4-4cc3-b1b2-75bc125bd5a8 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:30.893083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998865d-1dd4-4cc3-b1b2-75bc125bd5a8 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:30.902760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a7f1f-3f87-4c78-9829-83e57adc458c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:30.904680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a7f1f-3f87-4c78-9829-83e57adc458c admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:30.905445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a7f1f-3f87-4c78-9829-83e57adc458c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:30.905724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a7f1f-3f87-4c78-9829-83e57adc458c admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:30.906994 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a7f1f-3f87-4c78-9829-83e57adc458c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:30.911779 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a7f1f-3f87-4c78-9829-83e57adc458c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:31.006261 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-e2ba8f7b-0923-4300-aef9-9ae4f13fc1d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d5ec1ea62414eb5b76cb9135b0515c2'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:31.016728 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1100/2219] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:14:30 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:14:31.022749 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a7f1f-3f87-4c78-9829-83e57adc458c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:31.031639 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-e2ba8f7b-0923-4300-aef9-9ae4f13fc1d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d5ec1ea62414eb5b76cb9135b0515c2'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:31.037244 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:31.069100 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-e2ba8f7b-0923-4300-aef9-9ae4f13fc1d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d5ec1ea62414eb5b76cb9135b0515c2'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:31.071072 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-e2ba8f7b-0923-4300-aef9-9ae4f13fc1d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d5ec1ea62414eb5b76cb9135b0515c2'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:31.100568 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-e2ba8f7b-0923-4300-aef9-9ae4f13fc1d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d5ec1ea62414eb5b76cb9135b0515c2'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:31.121444 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1120/2220] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:14:30 2022] DELETE /identity/v3/users/7d5ec1ea62414eb5b76cb9135b0515c2 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:14:31.129343 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e3a8b-7da7-4805-8cec-ab4c57246707 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:31.129849 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e3a8b-7da7-4805-8cec-ab4c57246707 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:31.130371 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e3a8b-7da7-4805-8cec-ab4c57246707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:31.156756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:31.172278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-749e3a8b-7da7-4805-8cec-ab4c57246707 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:31.174024 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-749e3a8b-7da7-4805-8cec-ab4c57246707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:31.213724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a7f1f-3f87-4c78-9829-83e57adc458c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:31.338201 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1121/2221] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:14:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:31.348387 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b50b7-80f9-4699-9dc8-11ee7cdb0e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:31.349528 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b50b7-80f9-4699-9dc8-11ee7cdb0e34 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:31.350714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b50b7-80f9-4699-9dc8-11ee7cdb0e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:31.350714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b50b7-80f9-4699-9dc8-11ee7cdb0e34 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:31.351963 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b50b7-80f9-4699-9dc8-11ee7cdb0e34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:31.356129 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b50b7-80f9-4699-9dc8-11ee7cdb0e34 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:31.359431 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e3a8b-7da7-4805-8cec-ab4c57246707 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:31.362030 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-749e3a8b-7da7-4805-8cec-ab4c57246707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:31.393118 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1101/2222] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:14:31 2022] POST /identity/v3/users => generated 292 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:14:31.398212 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b50b7-80f9-4699-9dc8-11ee7cdb0e34 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:31.404393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-228b50b7-80f9-4699-9dc8-11ee7cdb0e34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:31.431358 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b50b7-80f9-4699-9dc8-11ee7cdb0e34 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:31.438625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857c85e-fb3a-459a-a8f0-6cb959cb4132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:31.439502 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857c85e-fb3a-459a-a8f0-6cb959cb4132 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:31.439987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857c85e-fb3a-459a-a8f0-6cb959cb4132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:31.440279 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857c85e-fb3a-459a-a8f0-6cb959cb4132 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:31.441331 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a857c85e-fb3a-459a-a8f0-6cb959cb4132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:31.444076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a857c85e-fb3a-459a-a8f0-6cb959cb4132 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:31.461318 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1102/2223] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:31.467515 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857c85e-fb3a-459a-a8f0-6cb959cb4132 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:31.485403 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857c85e-fb3a-459a-a8f0-6cb959cb4132 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:31.492208 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd8108-7714-414a-8b8d-9dde9832a6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:31.493344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd8108-7714-414a-8b8d-9dde9832a6df admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:31.493761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd8108-7714-414a-8b8d-9dde9832a6df admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:31.494234 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd8108-7714-414a-8b8d-9dde9832a6df admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:31.495480 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd8108-7714-414a-8b8d-9dde9832a6df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:31.498705 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd8108-7714-414a-8b8d-9dde9832a6df admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:31.500642 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e3a8b-7da7-4805-8cec-ab4c57246707 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:31.509444 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2de4a4-3220-49ef-bf17-8da3bd90adf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:31.510604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2de4a4-3220-49ef-bf17-8da3bd90adf5 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:31.511009 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2de4a4-3220-49ef-bf17-8da3bd90adf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:31.511522 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2de4a4-3220-49ef-bf17-8da3bd90adf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:31.525798 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1103/2224] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:14:31.527651 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd8108-7714-414a-8b8d-9dde9832a6df admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:31.536954 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2de4a4-3220-49ef-bf17-8da3bd90adf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:31.538432 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2de4a4-3220-49ef-bf17-8da3bd90adf5 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:31.553155 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd8108-7714-414a-8b8d-9dde9832a6df admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:31.557678 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1122/2225] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:14:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:14:31.567351 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8efc7-11f3-4d41-8a65-fe2961dc16d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:31.569041 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8efc7-11f3-4d41-8a65-fe2961dc16d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:31.569477 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8efc7-11f3-4d41-8a65-fe2961dc16d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:31.569831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8efc7-11f3-4d41-8a65-fe2961dc16d8 admin admin] PATH_INFO: `/v3/projects/24034448b386441c9389b3ac474611b0/users/2c2d80f49867464894f6523954710a36/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:31.600079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c8efc7-11f3-4d41-8a65-fe2961dc16d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24034448b386441c9389b3ac474611b0, user_id=2c2d80f49867464894f6523954710a36, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:31.608189 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c8efc7-11f3-4d41-8a65-fe2961dc16d8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:31.635435 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1104/2226] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:14:31 2022] PUT /identity/v3/projects/24034448b386441c9389b3ac474611b0/users/2c2d80f49867464894f6523954710a36/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:14:31.650360 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a7bda-559a-400c-869c-617ade393e91 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:31.651043 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a7bda-559a-400c-869c-617ade393e91 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:31.651621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a7bda-559a-400c-869c-617ade393e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:31.711105 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-f14a7bda-559a-400c-869c-617ade393e91 None None] MFA Rules not processed for user `2c2d80f49867464894f6523954710a36`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:31.714078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-f14a7bda-559a-400c-869c-617ade393e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:31.936616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1123/2227] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:14:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:31.959072 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8efc7-11f3-4d41-8a65-fe2961dc16d8 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:31.963873 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-86c8efc7-11f3-4d41-8a65-fe2961dc16d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:32.193847 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8efc7-11f3-4d41-8a65-fe2961dc16d8 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:32.200738 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61358c10-5205-4352-a0d7-791a81476416 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:32.201972 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61358c10-5205-4352-a0d7-791a81476416 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:32.201972 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61358c10-5205-4352-a0d7-791a81476416 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:32.202280 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61358c10-5205-4352-a0d7-791a81476416 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:32.217021 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61358c10-5205-4352-a0d7-791a81476416 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:32.218621 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61358c10-5205-4352-a0d7-791a81476416 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:32.239654 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1105/2228] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:14:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:14:32.325315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a7bda-559a-400c-869c-617ade393e91 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:32.327514 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-f14a7bda-559a-400c-869c-617ade393e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:32.394756 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:32.556384 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a7bda-559a-400c-869c-617ade393e91 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:32.566920 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee6a2a3-dd43-4257-8dee-89f660a6239b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:32.569107 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6a2a3-dd43-4257-8dee-89f660a6239b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:32.569713 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6a2a3-dd43-4257-8dee-89f660a6239b admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:32.570219 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6a2a3-dd43-4257-8dee-89f660a6239b admin admin] PATH_INFO: `/v3/projects/7b7043f964014dfc87f4f05a1412cec6` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:32.573346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee6a2a3-dd43-4257-8dee-89f660a6239b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b7043f964014dfc87f4f05a1412cec6)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:32.576071 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee6a2a3-dd43-4257-8dee-89f660a6239b admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:32.664496 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7ee6a2a3-dd43-4257-8dee-89f660a6239b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b7043f964014dfc87f4f05a1412cec6', 'request_id': 'req-7ee6a2a3-dd43-4257-8dee-89f660a6239b'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:32.672757 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7ee6a2a3-dd43-4257-8dee-89f660a6239b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b7043f964014dfc87f4f05a1412cec6', 'request_id': 'req-7ee6a2a3-dd43-4257-8dee-89f660a6239b'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:32.699530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7ee6a2a3-dd43-4257-8dee-89f660a6239b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b7043f964014dfc87f4f05a1412cec6', 'request_id': 'req-7ee6a2a3-dd43-4257-8dee-89f660a6239b'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:32.720733 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7ee6a2a3-dd43-4257-8dee-89f660a6239b admin admin] The token cache is being invalidate because project 7b7043f964014dfc87f4f05a1412cec6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:14:32.720733 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7ee6a2a3-dd43-4257-8dee-89f660a6239b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:32.721489 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7ee6a2a3-dd43-4257-8dee-89f660a6239b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:32.729837 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1124/2229] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:14:32 2022] DELETE /identity/v3/projects/7b7043f964014dfc87f4f05a1412cec6 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:32.743995 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1dc4e-7cea-4ed8-bee1-0c09b328b742 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:32.744773 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1dc4e-7cea-4ed8-bee1-0c09b328b742 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:32.745177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1dc4e-7cea-4ed8-bee1-0c09b328b742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:32.814817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9be1dc4e-7cea-4ed8-bee1-0c09b328b742 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:32.816676 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9be1dc4e-7cea-4ed8-bee1-0c09b328b742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:32.827609 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:32.984194 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1106/2230] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:14:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:32.996997 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8ed86-bb0d-4e5d-9b83-fc6147c2ecbc None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:32.998509 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8ed86-bb0d-4e5d-9b83-fc6147c2ecbc None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:32.999043 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8ed86-bb0d-4e5d-9b83-fc6147c2ecbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:33.041157 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-2aa8ed86-bb0d-4e5d-9b83-fc6147c2ecbc None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:33.044261 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-2aa8ed86-bb0d-4e5d-9b83-fc6147c2ecbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:33.086010 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1125/2231] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:14:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:33.091165 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1dc4e-7cea-4ed8-bee1-0c09b328b742 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:33.135457 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1dc4e-7cea-4ed8-bee1-0c09b328b742 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:33.148929 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb59750-579b-48cd-b8c1-33b9cc4c1689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:33.148929 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb59750-579b-48cd-b8c1-33b9cc4c1689 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:33.150327 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb59750-579b-48cd-b8c1-33b9cc4c1689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:33.150849 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb59750-579b-48cd-b8c1-33b9cc4c1689 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:33.152213 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb59750-579b-48cd-b8c1-33b9cc4c1689 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:14:33.152676 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb59750-579b-48cd-b8c1-33b9cc4c1689 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:33.155217 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb59750-579b-48cd-b8c1-33b9cc4c1689 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:33.180427 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1107/2232] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:14:33.182255 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8ed86-bb0d-4e5d-9b83-fc6147c2ecbc None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:33.201369 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8ed86-bb0d-4e5d-9b83-fc6147c2ecbc None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:33.207030 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f698f-7ab4-4569-a3ff-b42921f1fca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:33.207716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f698f-7ab4-4569-a3ff-b42921f1fca5 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:33.208039 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f698f-7ab4-4569-a3ff-b42921f1fca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:33.208314 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f698f-7ab4-4569-a3ff-b42921f1fca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:33.209739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f698f-7ab4-4569-a3ff-b42921f1fca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:33.212014 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f698f-7ab4-4569-a3ff-b42921f1fca5 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:33.257055 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1126/2233] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:14:33 2022] POST /identity/v3/projects => generated 385 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:14:33.263769 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb59750-579b-48cd-b8c1-33b9cc4c1689 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:33.279980 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb59750-579b-48cd-b8c1-33b9cc4c1689 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:33.354349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24e942-fb4f-42ba-99b0-fac5db5502ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:33.355543 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24e942-fb4f-42ba-99b0-fac5db5502ea admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:33.356110 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24e942-fb4f-42ba-99b0-fac5db5502ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:33.356635 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24e942-fb4f-42ba-99b0-fac5db5502ea admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:33.357656 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24e942-fb4f-42ba-99b0-fac5db5502ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:33.360105 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24e942-fb4f-42ba-99b0-fac5db5502ea admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:33.389165 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1108/2234] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:14:33 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:14:33.395165 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f698f-7ab4-4569-a3ff-b42921f1fca5 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:33.420924 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f698f-7ab4-4569-a3ff-b42921f1fca5 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:33.428132 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f461d5-831f-4ea3-9a0e-2894e6298017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:33.428933 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f461d5-831f-4ea3-9a0e-2894e6298017 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:33.429225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f461d5-831f-4ea3-9a0e-2894e6298017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:33.429572 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f461d5-831f-4ea3-9a0e-2894e6298017 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:33.430719 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f461d5-831f-4ea3-9a0e-2894e6298017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:33.433108 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f461d5-831f-4ea3-9a0e-2894e6298017 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:33.463966 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1127/2235] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:33.472347 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24e942-fb4f-42ba-99b0-fac5db5502ea admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:33.492495 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24e942-fb4f-42ba-99b0-fac5db5502ea admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:33.500121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa5df4-9104-40bb-9315-3b25518003a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:33.501297 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa5df4-9104-40bb-9315-3b25518003a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:33.501581 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa5df4-9104-40bb-9315-3b25518003a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:33.502161 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa5df4-9104-40bb-9315-3b25518003a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:33.503160 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa5df4-9104-40bb-9315-3b25518003a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:33.506262 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa5df4-9104-40bb-9315-3b25518003a7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:33.533573 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1109/2236] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:14:33.541358 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f461d5-831f-4ea3-9a0e-2894e6298017 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:33.579840 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f461d5-831f-4ea3-9a0e-2894e6298017 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:33.589320 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2fda1-d7c8-4720-b6e2-2abf7af2f9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:33.592236 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2fda1-d7c8-4720-b6e2-2abf7af2f9f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:33.592987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2fda1-d7c8-4720-b6e2-2abf7af2f9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:33.594346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2fda1-d7c8-4720-b6e2-2abf7af2f9f3 admin admin] PATH_INFO: `/v3/projects/925cc45674694931bfe8fc60765045c2/users/c0da50b34820422193be2b06e0da2837/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:33.625693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2fda1-d7c8-4720-b6e2-2abf7af2f9f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=925cc45674694931bfe8fc60765045c2, user_id=c0da50b34820422193be2b06e0da2837, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:33.632203 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2fda1-d7c8-4720-b6e2-2abf7af2f9f3 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:33.658596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1128/2237] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:14:33 2022] PUT /identity/v3/projects/925cc45674694931bfe8fc60765045c2/users/c0da50b34820422193be2b06e0da2837/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:33.667955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131999c2-4e85-4eb8-b44c-d7fb7b529f48 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:33.668524 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131999c2-4e85-4eb8-b44c-d7fb7b529f48 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:33.669008 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131999c2-4e85-4eb8-b44c-d7fb7b529f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:33.708047 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-131999c2-4e85-4eb8-b44c-d7fb7b529f48 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:33.710351 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-131999c2-4e85-4eb8-b44c-d7fb7b529f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:33.825761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1110/2238] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:14:33.837105 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2fda1-d7c8-4720-b6e2-2abf7af2f9f3 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:33.841099 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-cba2fda1-d7c8-4720-b6e2-2abf7af2f9f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:33.881741 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:33.919667 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2fda1-d7c8-4720-b6e2-2abf7af2f9f3 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:33.924047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cdeb2e-79df-4d11-9de3-2143dc5a79f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:33.924888 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cdeb2e-79df-4d11-9de3-2143dc5a79f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:33.925076 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cdeb2e-79df-4d11-9de3-2143dc5a79f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:33.925395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cdeb2e-79df-4d11-9de3-2143dc5a79f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:33.946230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cdeb2e-79df-4d11-9de3-2143dc5a79f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:33.948831 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cdeb2e-79df-4d11-9de3-2143dc5a79f0 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:33.965024 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1129/2239] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:14:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:14:34.377348 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001b90d-051c-4c51-860e-a533ddf64fd3 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:34.377956 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001b90d-051c-4c51-860e-a533ddf64fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:34.378505 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001b90d-051c-4c51-860e-a533ddf64fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:34.409099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f001b90d-051c-4c51-860e-a533ddf64fd3 None None] MFA Rules not processed for user `246ac84c69894c08b870c4579cabd042`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:34.410950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f001b90d-051c-4c51-860e-a533ddf64fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:34.437644 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:34.608960 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1111/2240] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:14:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:14:35.668883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cdeb2e-79df-4d11-9de3-2143dc5a79f0 service neutron] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:35.670661 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-28cdeb2e-79df-4d11-9de3-2143dc5a79f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:35.687935 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cdeb2e-79df-4d11-9de3-2143dc5a79f0 service neutron] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:35.694396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a14d02-9406-426d-bd1c-73480927ec67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:35.696263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a14d02-9406-426d-bd1c-73480927ec67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:35.696860 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a14d02-9406-426d-bd1c-73480927ec67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:35.698017 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a14d02-9406-426d-bd1c-73480927ec67 admin admin] PATH_INFO: `/v3/users/2c2d80f49867464894f6523954710a36` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:35.699585 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a14d02-9406-426d-bd1c-73480927ec67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c2d80f49867464894f6523954710a36)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:35.701800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a14d02-9406-426d-bd1c-73480927ec67 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:35.817173 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-65a14d02-9406-426d-bd1c-73480927ec67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c2d80f49867464894f6523954710a36'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:35.832255 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-65a14d02-9406-426d-bd1c-73480927ec67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c2d80f49867464894f6523954710a36'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:35.887972 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-65a14d02-9406-426d-bd1c-73480927ec67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c2d80f49867464894f6523954710a36'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:35.890055 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-65a14d02-9406-426d-bd1c-73480927ec67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c2d80f49867464894f6523954710a36'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:35.906219 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-65a14d02-9406-426d-bd1c-73480927ec67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c2d80f49867464894f6523954710a36'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:35.927180 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1130/2241] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:14:35 2022] DELETE /identity/v3/users/2c2d80f49867464894f6523954710a36 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:35.934485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b7014-ece8-433f-bbee-8b4f83ec7036 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:35.935430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b7014-ece8-433f-bbee-8b4f83ec7036 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:35.936240 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b7014-ece8-433f-bbee-8b4f83ec7036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:35.977785 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-937b7014-ece8-433f-bbee-8b4f83ec7036 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:35.980264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-937b7014-ece8-433f-bbee-8b4f83ec7036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:36.119682 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1112/2242] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:14:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:14:36.133510 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a14d02-9406-426d-bd1c-73480927ec67 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:36.136607 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-65a14d02-9406-426d-bd1c-73480927ec67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:36.249108 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a14d02-9406-426d-bd1c-73480927ec67 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:36.262349 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1c6df-a6b7-4599-a93d-a675e742058a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:36.263044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1c6df-a6b7-4599-a93d-a675e742058a service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:36.263432 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1c6df-a6b7-4599-a93d-a675e742058a service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:36.263721 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1c6df-a6b7-4599-a93d-a675e742058a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:36.317135 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1c6df-a6b7-4599-a93d-a675e742058a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:36.320335 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1c6df-a6b7-4599-a93d-a675e742058a service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:36.346772 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1131/2243] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:14:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:14:37.406470 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b7014-ece8-433f-bbee-8b4f83ec7036 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:37.408594 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-937b7014-ece8-433f-bbee-8b4f83ec7036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:37.434473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b7014-ece8-433f-bbee-8b4f83ec7036 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:37.442233 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb2fd5-8c16-4899-b340-c26d7567a01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:37.443479 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb2fd5-8c16-4899-b340-c26d7567a01d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:37.443921 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb2fd5-8c16-4899-b340-c26d7567a01d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:37.444348 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb2fd5-8c16-4899-b340-c26d7567a01d admin admin] PATH_INFO: `/v3/projects/24034448b386441c9389b3ac474611b0` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:37.447799 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cb2fd5-8c16-4899-b340-c26d7567a01d admin admin] RBAC: Authorizing `identity:delete_project(project_id=24034448b386441c9389b3ac474611b0)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:37.450392 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cb2fd5-8c16-4899-b340-c26d7567a01d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:37.569982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-36cb2fd5-8c16-4899-b340-c26d7567a01d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24034448b386441c9389b3ac474611b0', 'request_id': 'req-36cb2fd5-8c16-4899-b340-c26d7567a01d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:37.587505 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-36cb2fd5-8c16-4899-b340-c26d7567a01d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24034448b386441c9389b3ac474611b0', 'request_id': 'req-36cb2fd5-8c16-4899-b340-c26d7567a01d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:37.609161 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-36cb2fd5-8c16-4899-b340-c26d7567a01d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24034448b386441c9389b3ac474611b0', 'request_id': 'req-36cb2fd5-8c16-4899-b340-c26d7567a01d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:37.629539 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-36cb2fd5-8c16-4899-b340-c26d7567a01d admin admin] The token cache is being invalidate because project 24034448b386441c9389b3ac474611b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:14:37.630162 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-36cb2fd5-8c16-4899-b340-c26d7567a01d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:37.631303 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-36cb2fd5-8c16-4899-b340-c26d7567a01d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:37.639890 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1113/2244] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:14:37 2022] DELETE /identity/v3/projects/24034448b386441c9389b3ac474611b0 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:37.646810 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89e9fe-4c23-4045-91ff-18d513e3a3cf None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:37.647338 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89e9fe-4c23-4045-91ff-18d513e3a3cf None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:37.647702 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89e9fe-4c23-4045-91ff-18d513e3a3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:37.691497 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-4c89e9fe-4c23-4045-91ff-18d513e3a3cf None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:37.693063 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-4c89e9fe-4c23-4045-91ff-18d513e3a3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:37.796533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:37.843075 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1132/2245] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:37.857231 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db6ad3-9767-40b4-8ea2-61de47d6ed1c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:37.857688 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db6ad3-9767-40b4-8ea2-61de47d6ed1c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:37.858183 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db6ad3-9767-40b4-8ea2-61de47d6ed1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:37.902006 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c8db6ad3-9767-40b4-8ea2-61de47d6ed1c None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:37.904878 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c8db6ad3-9767-40b4-8ea2-61de47d6ed1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:37.923855 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1114/2246] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:37.931606 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89e9fe-4c23-4045-91ff-18d513e3a3cf None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:37.964649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89e9fe-4c23-4045-91ff-18d513e3a3cf None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:37.974541 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c1895-5d14-4e89-a760-6c354b578c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:37.976134 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c1895-5d14-4e89-a760-6c354b578c23 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:37.976687 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c1895-5d14-4e89-a760-6c354b578c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:37.977673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c1895-5d14-4e89-a760-6c354b578c23 admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:37.984551 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c1895-5d14-4e89-a760-6c354b578c23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:14:37.984551 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c1895-5d14-4e89-a760-6c354b578c23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:37.984551 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c1895-5d14-4e89-a760-6c354b578c23 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:38.018597 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1133/2247] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:14:38.023348 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db6ad3-9767-40b4-8ea2-61de47d6ed1c None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:38.043151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db6ad3-9767-40b4-8ea2-61de47d6ed1c None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:38.059676 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e0c41-4092-42c8-a4ee-5b1a8ce399a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:38.060474 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e0c41-4092-42c8-a4ee-5b1a8ce399a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:38.060964 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e0c41-4092-42c8-a4ee-5b1a8ce399a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:38.061251 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e0c41-4092-42c8-a4ee-5b1a8ce399a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:38.062179 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8e0c41-4092-42c8-a4ee-5b1a8ce399a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:38.064192 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8e0c41-4092-42c8-a4ee-5b1a8ce399a6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:38.154639 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c1895-5d14-4e89-a760-6c354b578c23 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:38.162737 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1115/2248] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:14:38 2022] POST /identity/v3/projects => generated 365 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:14:38.182255 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c1895-5d14-4e89-a760-6c354b578c23 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:38.329379 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cecdeb-922c-4a3f-ab1a-d36d3f91a2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:38.330120 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cecdeb-922c-4a3f-ab1a-d36d3f91a2f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:38.330533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cecdeb-922c-4a3f-ab1a-d36d3f91a2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:38.331151 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cecdeb-922c-4a3f-ab1a-d36d3f91a2f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:38.331793 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cecdeb-922c-4a3f-ab1a-d36d3f91a2f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:38.335535 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cecdeb-922c-4a3f-ab1a-d36d3f91a2f6 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:38.369249 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1134/2249] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:14:38 2022] POST /identity/v3/users => generated 283 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:14:38.374445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e0c41-4092-42c8-a4ee-5b1a8ce399a6 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:38.392361 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e0c41-4092-42c8-a4ee-5b1a8ce399a6 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:38.400272 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba56868b-58fb-46b3-acb3-22f17464105b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:38.401123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56868b-58fb-46b3-acb3-22f17464105b admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:38.401597 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56868b-58fb-46b3-acb3-22f17464105b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:38.401973 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56868b-58fb-46b3-acb3-22f17464105b admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:38.402922 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba56868b-58fb-46b3-acb3-22f17464105b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:38.405195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba56868b-58fb-46b3-acb3-22f17464105b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:38.426953 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1116/2250] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:38.433678 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cecdeb-922c-4a3f-ab1a-d36d3f91a2f6 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:38.454738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cecdeb-922c-4a3f-ab1a-d36d3f91a2f6 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:38.461518 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d619b-36e9-4b7b-bea0-b8a445346733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:38.463361 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d619b-36e9-4b7b-bea0-b8a445346733 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:38.463361 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d619b-36e9-4b7b-bea0-b8a445346733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:38.463361 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d619b-36e9-4b7b-bea0-b8a445346733 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:38.463361 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d619b-36e9-4b7b-bea0-b8a445346733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:38.468766 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d619b-36e9-4b7b-bea0-b8a445346733 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:38.490737 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1135/2251] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:38.498600 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba56868b-58fb-46b3-acb3-22f17464105b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:38.507290 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:38.531468 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba56868b-58fb-46b3-acb3-22f17464105b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:38.544783 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f667aa-b5af-460d-8113-9980145fb6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:38.545756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f667aa-b5af-460d-8113-9980145fb6d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:38.546253 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f667aa-b5af-460d-8113-9980145fb6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:38.546709 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f667aa-b5af-460d-8113-9980145fb6d6 admin admin] PATH_INFO: `/v3/projects/8ee8a35d0ab2411488227c7e2084be97/users/e0d2811ee47a427ab91069efbbc5806c/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:38.598540 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f667aa-b5af-460d-8113-9980145fb6d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ee8a35d0ab2411488227c7e2084be97, user_id=e0d2811ee47a427ab91069efbbc5806c, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:38.598540 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f667aa-b5af-460d-8113-9980145fb6d6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:38.634350 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1117/2252] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:14:38 2022] PUT /identity/v3/projects/8ee8a35d0ab2411488227c7e2084be97/users/e0d2811ee47a427ab91069efbbc5806c/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:38.642584 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914baa97-5aea-4381-9d40-2c861761da0c None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:38.643112 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914baa97-5aea-4381-9d40-2c861761da0c None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:38.643645 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914baa97-5aea-4381-9d40-2c861761da0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:38.712987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-914baa97-5aea-4381-9d40-2c861761da0c None None] MFA Rules not processed for user `e0d2811ee47a427ab91069efbbc5806c`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:38.715127 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-914baa97-5aea-4381-9d40-2c861761da0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:38.861777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1136/2253] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:14:38 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:14:38.876331 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f667aa-b5af-460d-8113-9980145fb6d6 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:38.878848 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-45f667aa-b5af-460d-8113-9980145fb6d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:39.048202 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f667aa-b5af-460d-8113-9980145fb6d6 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:39.053681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0a186-aa49-4bc3-96c6-297255e78bf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:39.054445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0a186-aa49-4bc3-96c6-297255e78bf4 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:39.054703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0a186-aa49-4bc3-96c6-297255e78bf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:39.055825 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0a186-aa49-4bc3-96c6-297255e78bf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:39.069089 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0a186-aa49-4bc3-96c6-297255e78bf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:39.070333 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0a186-aa49-4bc3-96c6-297255e78bf4 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:39.087992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1118/2254] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:14:38 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:14:41.326158 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914baa97-5aea-4381-9d40-2c861761da0c None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:41.331359 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-914baa97-5aea-4381-9d40-2c861761da0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:41.576837 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:41.676250 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914baa97-5aea-4381-9d40-2c861761da0c None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:41.689871 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90200fdc-e66d-4c5b-9363-709f1bfcc01c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36b63614297346e29ff276eab444fe5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:41.689871 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90200fdc-e66d-4c5b-9363-709f1bfcc01c service placement] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:41.689871 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90200fdc-e66d-4c5b-9363-709f1bfcc01c service placement] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:41.689871 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90200fdc-e66d-4c5b-9363-709f1bfcc01c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:41.696268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-90200fdc-e66d-4c5b-9363-709f1bfcc01c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:41.721341 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90200fdc-e66d-4c5b-9363-709f1bfcc01c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:41.724703 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90200fdc-e66d-4c5b-9363-709f1bfcc01c service placement] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:41.749549 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1137/2255] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:14:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:14:43.415233 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028e01d-96ac-4a82-a0f1-73f6039616c1 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:43.416620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028e01d-96ac-4a82-a0f1-73f6039616c1 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:43.418902 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028e01d-96ac-4a82-a0f1-73f6039616c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:43.527340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e028e01d-96ac-4a82-a0f1-73f6039616c1 None None] MFA Rules not processed for user `c0da50b34820422193be2b06e0da2837`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:43.529267 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e028e01d-96ac-4a82-a0f1-73f6039616c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:43.797213 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1119/2256] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:14:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:14:43.811186 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90200fdc-e66d-4c5b-9363-709f1bfcc01c service placement] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:43.815303 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-90200fdc-e66d-4c5b-9363-709f1bfcc01c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:43.946492 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90200fdc-e66d-4c5b-9363-709f1bfcc01c service placement] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:43.952341 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c3708-2dbd-4314-8a26-db21f014d2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:43.954009 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c3708-2dbd-4314-8a26-db21f014d2c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:43.954670 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c3708-2dbd-4314-8a26-db21f014d2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:43.955368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c3708-2dbd-4314-8a26-db21f014d2c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:43.956763 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526c3708-2dbd-4314-8a26-db21f014d2c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:43.961128 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526c3708-2dbd-4314-8a26-db21f014d2c9 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:44.037600 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1138/2257] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:14:43 2022] POST /identity/v3/projects => generated 383 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:14:44.056217 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028e01d-96ac-4a82-a0f1-73f6039616c1 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:44.095497 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028e01d-96ac-4a82-a0f1-73f6039616c1 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:44.278230 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f906bd-f9bf-41e3-8182-91c218ed9fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:44.279822 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f906bd-f9bf-41e3-8182-91c218ed9fc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:44.280387 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f906bd-f9bf-41e3-8182-91c218ed9fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:44.280881 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f906bd-f9bf-41e3-8182-91c218ed9fc0 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:44.282291 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f906bd-f9bf-41e3-8182-91c218ed9fc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:44.286351 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f906bd-f9bf-41e3-8182-91c218ed9fc0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:44.328223 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1120/2258] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:14:44 2022] POST /identity/v3/users => generated 298 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:14:44.329627 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c3708-2dbd-4314-8a26-db21f014d2c9 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:44.347222 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c3708-2dbd-4314-8a26-db21f014d2c9 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:44.371847 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05692086-a6b1-4301-b78a-67d3ba1f3bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:44.375098 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05692086-a6b1-4301-b78a-67d3ba1f3bcc admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:44.376748 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05692086-a6b1-4301-b78a-67d3ba1f3bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:44.377490 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05692086-a6b1-4301-b78a-67d3ba1f3bcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:44.382163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05692086-a6b1-4301-b78a-67d3ba1f3bcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:44.385917 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05692086-a6b1-4301-b78a-67d3ba1f3bcc admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:44.412388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1139/2259] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:44.413740 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f906bd-f9bf-41e3-8182-91c218ed9fc0 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:44.452778 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:44.465392 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f906bd-f9bf-41e3-8182-91c218ed9fc0 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:44.488745 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c8e01-5c6c-4b73-8b44-40747537b35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:44.488745 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c8e01-5c6c-4b73-8b44-40747537b35d admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:44.488745 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c8e01-5c6c-4b73-8b44-40747537b35d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:44.488745 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c8e01-5c6c-4b73-8b44-40747537b35d admin admin] PATH_INFO: `/v3/projects/e9d65bd6f7e241859054c2ab08fa9775/users/d5acd259bd7648aba86df057ea081655/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:44.511652 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c8e01-5c6c-4b73-8b44-40747537b35d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9d65bd6f7e241859054c2ab08fa9775, user_id=d5acd259bd7648aba86df057ea081655, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:44.537980 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c8e01-5c6c-4b73-8b44-40747537b35d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:44.576681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1121/2260] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:14:44 2022] PUT /identity/v3/projects/e9d65bd6f7e241859054c2ab08fa9775/users/d5acd259bd7648aba86df057ea081655/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:14:44.579367 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05692086-a6b1-4301-b78a-67d3ba1f3bcc admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:44.607877 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05692086-a6b1-4301-b78a-67d3ba1f3bcc admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:44.767995 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c8e01-5c6c-4b73-8b44-40747537b35d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:44.796208 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2f2c8e01-5c6c-4b73-8b44-40747537b35d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:44.873334 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01bd19-74e4-4bf6-83cd-2a5904a549a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:44.878851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01bd19-74e4-4bf6-83cd-2a5904a549a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:44.879333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01bd19-74e4-4bf6-83cd-2a5904a549a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:44.883030 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01bd19-74e4-4bf6-83cd-2a5904a549a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:44.886435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01bd19-74e4-4bf6-83cd-2a5904a549a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:44.889108 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01bd19-74e4-4bf6-83cd-2a5904a549a4 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:44.919335 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1140/2261] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:44.926085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01bd19-74e4-4bf6-83cd-2a5904a549a4 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:44.964305 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01bd19-74e4-4bf6-83cd-2a5904a549a4 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:44.985061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba7894-90c4-46fd-90ae-17795221fa61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:44.986937 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba7894-90c4-46fd-90ae-17795221fa61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:44.987709 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba7894-90c4-46fd-90ae-17795221fa61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:44.988476 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba7894-90c4-46fd-90ae-17795221fa61 admin admin] PATH_INFO: `/v3/projects/e9d65bd6f7e241859054c2ab08fa9775/users/d5acd259bd7648aba86df057ea081655/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:45.001785 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcba7894-90c4-46fd-90ae-17795221fa61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9d65bd6f7e241859054c2ab08fa9775, user_id=d5acd259bd7648aba86df057ea081655, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:45.006399 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcba7894-90c4-46fd-90ae-17795221fa61 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:45.029379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c8e01-5c6c-4b73-8b44-40747537b35d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:45.057818 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1141/2262] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:14:44 2022] PUT /identity/v3/projects/e9d65bd6f7e241859054c2ab08fa9775/users/d5acd259bd7648aba86df057ea081655/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:45.214841 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ac734-d607-40d1-be74-5036771b2b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:45.217592 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ac734-d607-40d1-be74-5036771b2b91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:45.218405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ac734-d607-40d1-be74-5036771b2b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:45.218991 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ac734-d607-40d1-be74-5036771b2b91 admin admin] PATH_INFO: `/v3/users/5573930262ff4d7988e0124b27fbf7e8` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:45.221769 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ac734-d607-40d1-be74-5036771b2b91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5573930262ff4d7988e0124b27fbf7e8)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:45.226228 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ac734-d607-40d1-be74-5036771b2b91 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:45.332479 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-293ac734-d607-40d1-be74-5036771b2b91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5573930262ff4d7988e0124b27fbf7e8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:45.352035 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-293ac734-d607-40d1-be74-5036771b2b91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5573930262ff4d7988e0124b27fbf7e8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:45.368495 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-293ac734-d607-40d1-be74-5036771b2b91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5573930262ff4d7988e0124b27fbf7e8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:45.380686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-293ac734-d607-40d1-be74-5036771b2b91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5573930262ff4d7988e0124b27fbf7e8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:45.381748 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-293ac734-d607-40d1-be74-5036771b2b91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5573930262ff4d7988e0124b27fbf7e8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:45.391085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1122/2263] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:14:44 2022] DELETE /identity/v3/users/5573930262ff4d7988e0124b27fbf7e8 => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:45.395812 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b4b39-acb6-4bb9-9573-b1cdd4f61ee4 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:45.396171 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b4b39-acb6-4bb9-9573-b1cdd4f61ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:45.396597 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b4b39-acb6-4bb9-9573-b1cdd4f61ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:45.422479 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-a69b4b39-acb6-4bb9-9573-b1cdd4f61ee4 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:45.424106 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-a69b4b39-acb6-4bb9-9573-b1cdd4f61ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:45.440165 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:45.546583 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1142/2264] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:14:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:45.562393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ac734-d607-40d1-be74-5036771b2b91 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:45.564141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-293ac734-d607-40d1-be74-5036771b2b91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:45.584223 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:45.719877 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ac734-d607-40d1-be74-5036771b2b91 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:45.727591 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c84db-2151-49e2-b7e9-dd91819ebc5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:45.727902 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c84db-2151-49e2-b7e9-dd91819ebc5a service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:45.728268 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c84db-2151-49e2-b7e9-dd91819ebc5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:45.728681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c84db-2151-49e2-b7e9-dd91819ebc5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:45.745493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c84db-2151-49e2-b7e9-dd91819ebc5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:45.750324 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c84db-2151-49e2-b7e9-dd91819ebc5a service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:45.770686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1123/2265] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:14:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:14:47.421533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b4b39-acb6-4bb9-9573-b1cdd4f61ee4 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:47.424557 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-a69b4b39-acb6-4bb9-9573-b1cdd4f61ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:47.464387 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b4b39-acb6-4bb9-9573-b1cdd4f61ee4 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:47.486734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38faad50-25ff-40a1-913d-f4b85fc10fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:47.489063 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38faad50-25ff-40a1-913d-f4b85fc10fbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:47.490232 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38faad50-25ff-40a1-913d-f4b85fc10fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:47.492644 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38faad50-25ff-40a1-913d-f4b85fc10fbe admin admin] PATH_INFO: `/v3/projects/b3c03ed4e63b41958094fb3315a04d1e` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:47.502325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38faad50-25ff-40a1-913d-f4b85fc10fbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3c03ed4e63b41958094fb3315a04d1e)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:47.514453 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38faad50-25ff-40a1-913d-f4b85fc10fbe admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:47.699842 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-38faad50-25ff-40a1-913d-f4b85fc10fbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3c03ed4e63b41958094fb3315a04d1e', 'request_id': 'req-38faad50-25ff-40a1-913d-f4b85fc10fbe'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:47.714038 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-38faad50-25ff-40a1-913d-f4b85fc10fbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3c03ed4e63b41958094fb3315a04d1e', 'request_id': 'req-38faad50-25ff-40a1-913d-f4b85fc10fbe'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:47.729783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-38faad50-25ff-40a1-913d-f4b85fc10fbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3c03ed4e63b41958094fb3315a04d1e', 'request_id': 'req-38faad50-25ff-40a1-913d-f4b85fc10fbe'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:47.772112 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-38faad50-25ff-40a1-913d-f4b85fc10fbe admin admin] The token cache is being invalidate because project b3c03ed4e63b41958094fb3315a04d1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:14:47.772112 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-38faad50-25ff-40a1-913d-f4b85fc10fbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:47.775109 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-38faad50-25ff-40a1-913d-f4b85fc10fbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:47.781599 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1143/2266] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:14:47 2022] DELETE /identity/v3/projects/b3c03ed4e63b41958094fb3315a04d1e => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:47.789895 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c84db-2151-49e2-b7e9-dd91819ebc5a service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:47.818214 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c84db-2151-49e2-b7e9-dd91819ebc5a service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:48.000229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc83f13-e729-461b-97e4-2805c0dcfcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:48.001329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc83f13-e729-461b-97e4-2805c0dcfcca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:48.002890 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc83f13-e729-461b-97e4-2805c0dcfcca admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:48.003186 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc83f13-e729-461b-97e4-2805c0dcfcca admin admin] PATH_INFO: `/v3/users/017b42a077174663939a26ddee024222` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:48.006307 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc83f13-e729-461b-97e4-2805c0dcfcca admin admin] RBAC: Authorizing `identity:delete_user(user_id=017b42a077174663939a26ddee024222)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:48.012897 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc83f13-e729-461b-97e4-2805c0dcfcca admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:48.219160 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3cc83f13-e729-461b-97e4-2805c0dcfcca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '017b42a077174663939a26ddee024222'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:48.273118 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3cc83f13-e729-461b-97e4-2805c0dcfcca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '017b42a077174663939a26ddee024222'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:48.295259 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3cc83f13-e729-461b-97e4-2805c0dcfcca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '017b42a077174663939a26ddee024222'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:48.318240 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3cc83f13-e729-461b-97e4-2805c0dcfcca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '017b42a077174663939a26ddee024222'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:48.322226 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3cc83f13-e729-461b-97e4-2805c0dcfcca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '017b42a077174663939a26ddee024222'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:48.344067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1124/2267] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:14:47 2022] DELETE /identity/v3/users/017b42a077174663939a26ddee024222 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:50.263203 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71f8b7-e648-4f8d-bf40-1a212b2ed1e5 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:50.263737 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71f8b7-e648-4f8d-bf40-1a212b2ed1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:50.264254 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71f8b7-e648-4f8d-bf40-1a212b2ed1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:50.375102 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-8e71f8b7-e648-4f8d-bf40-1a212b2ed1e5 None None] MFA Rules not processed for user `2c34f05634d34da29088bcf75069958d`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:50.380377 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-8e71f8b7-e648-4f8d-bf40-1a212b2ed1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:50.585039 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1144/2268] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:14:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:51.372234 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc83f13-e729-461b-97e4-2805c0dcfcca admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:51.372234 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3cc83f13-e729-461b-97e4-2805c0dcfcca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:51.474616 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:51.545590 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc83f13-e729-461b-97e4-2805c0dcfcca admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:51.557360 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c80be4e-47e1-4ac5-97fd-06d044e09565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:51.559521 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80be4e-47e1-4ac5-97fd-06d044e09565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:51.560429 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80be4e-47e1-4ac5-97fd-06d044e09565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:51.561574 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80be4e-47e1-4ac5-97fd-06d044e09565 admin admin] PATH_INFO: `/v3/projects/cbf8ddcc1b344751a1cde419f7394d46` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:51.565238 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c80be4e-47e1-4ac5-97fd-06d044e09565 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbf8ddcc1b344751a1cde419f7394d46)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:51.567976 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c80be4e-47e1-4ac5-97fd-06d044e09565 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:51.649159 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9c80be4e-47e1-4ac5-97fd-06d044e09565 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbf8ddcc1b344751a1cde419f7394d46', 'request_id': 'req-9c80be4e-47e1-4ac5-97fd-06d044e09565'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:51.658051 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9c80be4e-47e1-4ac5-97fd-06d044e09565 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbf8ddcc1b344751a1cde419f7394d46', 'request_id': 'req-9c80be4e-47e1-4ac5-97fd-06d044e09565'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:51.673502 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9c80be4e-47e1-4ac5-97fd-06d044e09565 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbf8ddcc1b344751a1cde419f7394d46', 'request_id': 'req-9c80be4e-47e1-4ac5-97fd-06d044e09565'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:51.699383 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9c80be4e-47e1-4ac5-97fd-06d044e09565 admin admin] The token cache is being invalidate because project cbf8ddcc1b344751a1cde419f7394d46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:14:51.699877 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9c80be4e-47e1-4ac5-97fd-06d044e09565 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:51.700690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9c80be4e-47e1-4ac5-97fd-06d044e09565 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:51.703836 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1125/2269] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:14:51 2022] DELETE /identity/v3/projects/cbf8ddcc1b344751a1cde419f7394d46 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:51.713734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bfe78-6b07-4750-b3ca-6e5aac285eb3 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:51.714339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bfe78-6b07-4750-b3ca-6e5aac285eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:51.715623 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bfe78-6b07-4750-b3ca-6e5aac285eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:51.769646 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-fe8bfe78-6b07-4750-b3ca-6e5aac285eb3 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:51.772235 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-fe8bfe78-6b07-4750-b3ca-6e5aac285eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:51.849895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:51.894448 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1145/2270] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:14:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:51.905037 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a7b29-8db7-42ac-8f95-4c5a8299245f None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:51.905970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a7b29-8db7-42ac-8f95-4c5a8299245f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:51.905970 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a7b29-8db7-42ac-8f95-4c5a8299245f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:51.933594 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9c6a7b29-8db7-42ac-8f95-4c5a8299245f None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:51.935086 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9c6a7b29-8db7-42ac-8f95-4c5a8299245f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:51.969506 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1126/2271] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:14:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:51.974192 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8bfe78-6b07-4750-b3ca-6e5aac285eb3 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:51.994123 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8bfe78-6b07-4750-b3ca-6e5aac285eb3 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:52.003166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811a173-4a54-473b-afd6-776aa0752097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:52.004042 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811a173-4a54-473b-afd6-776aa0752097 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:52.004335 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811a173-4a54-473b-afd6-776aa0752097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:52.004983 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811a173-4a54-473b-afd6-776aa0752097 admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:52.006358 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f811a173-4a54-473b-afd6-776aa0752097 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:14:52.006848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f811a173-4a54-473b-afd6-776aa0752097 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:52.009504 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f811a173-4a54-473b-afd6-776aa0752097 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:52.027744 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1146/2272] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:14:52.031944 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a7b29-8db7-42ac-8f95-4c5a8299245f None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:52.059828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a7b29-8db7-42ac-8f95-4c5a8299245f None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:52.067175 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6304c30-2f3e-4c6c-960e-9b74b77f5135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:52.068130 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6304c30-2f3e-4c6c-960e-9b74b77f5135 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:52.068533 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6304c30-2f3e-4c6c-960e-9b74b77f5135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:52.069605 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6304c30-2f3e-4c6c-960e-9b74b77f5135 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:52.072173 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6304c30-2f3e-4c6c-960e-9b74b77f5135 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:52.074589 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6304c30-2f3e-4c6c-960e-9b74b77f5135 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:52.147813 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1127/2273] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:14:52 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:14:52.152596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811a173-4a54-473b-afd6-776aa0752097 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:52.173137 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811a173-4a54-473b-afd6-776aa0752097 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:52.370683 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c219a1d-847d-4bbd-8ce9-1fd28d714fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:52.371596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c219a1d-847d-4bbd-8ce9-1fd28d714fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:52.371987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c219a1d-847d-4bbd-8ce9-1fd28d714fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:52.372412 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c219a1d-847d-4bbd-8ce9-1fd28d714fc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:52.375167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c219a1d-847d-4bbd-8ce9-1fd28d714fc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:52.375700 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c219a1d-847d-4bbd-8ce9-1fd28d714fc7 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:52.416077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1147/2274] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:14:52 2022] POST /identity/v3/users => generated 293 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:14:52.422465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6304c30-2f3e-4c6c-960e-9b74b77f5135 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:52.461510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6304c30-2f3e-4c6c-960e-9b74b77f5135 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:52.516853 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97526b-9fe9-4b87-803a-9afbe0dbd54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:52.521234 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97526b-9fe9-4b87-803a-9afbe0dbd54d admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:52.522013 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97526b-9fe9-4b87-803a-9afbe0dbd54d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:52.522734 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97526b-9fe9-4b87-803a-9afbe0dbd54d admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:52.525222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97526b-9fe9-4b87-803a-9afbe0dbd54d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:52.530286 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97526b-9fe9-4b87-803a-9afbe0dbd54d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:52.610966 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1128/2275] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:52.613585 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c219a1d-847d-4bbd-8ce9-1fd28d714fc7 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:52.651401 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c219a1d-847d-4bbd-8ce9-1fd28d714fc7 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:52.677814 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba01829d-41b0-45a2-b5bb-c856e90ed946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:52.678556 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01829d-41b0-45a2-b5bb-c856e90ed946 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:52.678844 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01829d-41b0-45a2-b5bb-c856e90ed946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:52.679122 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01829d-41b0-45a2-b5bb-c856e90ed946 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:52.680144 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba01829d-41b0-45a2-b5bb-c856e90ed946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:52.683279 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba01829d-41b0-45a2-b5bb-c856e90ed946 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:52.722087 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97526b-9fe9-4b87-803a-9afbe0dbd54d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:52.722523 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1148/2276] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:52.743678 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97526b-9fe9-4b87-803a-9afbe0dbd54d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:52.755972 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e53b69a-2b45-427c-8845-c8778c7bcc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:52.756901 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53b69a-2b45-427c-8845-c8778c7bcc7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:52.757189 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53b69a-2b45-427c-8845-c8778c7bcc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:52.757470 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53b69a-2b45-427c-8845-c8778c7bcc7b admin admin] PATH_INFO: `/v3/projects/0ccbf228e6dc4df19b0c488297f36ada/users/882660f949ff4bce837eace270d48d53/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:52.790507 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e53b69a-2b45-427c-8845-c8778c7bcc7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ccbf228e6dc4df19b0c488297f36ada, user_id=882660f949ff4bce837eace270d48d53, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:52.793745 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e53b69a-2b45-427c-8845-c8778c7bcc7b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:52.833236 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1129/2277] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:14:52 2022] PUT /identity/v3/projects/0ccbf228e6dc4df19b0c488297f36ada/users/882660f949ff4bce837eace270d48d53/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:52.847449 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfbf35-77e8-4d14-a8af-94b909ee149f None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:52.847942 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfbf35-77e8-4d14-a8af-94b909ee149f None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:52.848635 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfbf35-77e8-4d14-a8af-94b909ee149f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:52.947588 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-c8cfbf35-77e8-4d14-a8af-94b909ee149f None None] MFA Rules not processed for user `882660f949ff4bce837eace270d48d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:52.964756 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-c8cfbf35-77e8-4d14-a8af-94b909ee149f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:53.192567 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1149/2278] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:14:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:53.199861 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e53b69a-2b45-427c-8845-c8778c7bcc7b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:53.344538 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:53.379824 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e53b69a-2b45-427c-8845-c8778c7bcc7b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:53.390535 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6442d17-496d-40c5-a9ce-c4656d96866b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:53.392837 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6442d17-496d-40c5-a9ce-c4656d96866b admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:53.393712 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6442d17-496d-40c5-a9ce-c4656d96866b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:53.394547 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6442d17-496d-40c5-a9ce-c4656d96866b admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:53.396264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6442d17-496d-40c5-a9ce-c4656d96866b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:53.402585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6442d17-496d-40c5-a9ce-c4656d96866b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:53.468164 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1130/2279] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:14:53 2022] POST /identity/v3/projects => generated 387 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:14:53.472972 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cfbf35-77e8-4d14-a8af-94b909ee149f None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:53.488966 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cfbf35-77e8-4d14-a8af-94b909ee149f None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:53.543816 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:53.589543 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8dc2d0-27fb-411d-8ed7-71e929a2b49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:53.590127 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8dc2d0-27fb-411d-8ed7-71e929a2b49b admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:53.590732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8dc2d0-27fb-411d-8ed7-71e929a2b49b admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:53.591555 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8dc2d0-27fb-411d-8ed7-71e929a2b49b admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:53.592334 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8dc2d0-27fb-411d-8ed7-71e929a2b49b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:53.594944 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8dc2d0-27fb-411d-8ed7-71e929a2b49b admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:53.632404 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1150/2280] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:14:53 2022] POST /identity/v3/users => generated 294 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:14:53.637700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6442d17-496d-40c5-a9ce-c4656d96866b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:53.649652 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6442d17-496d-40c5-a9ce-c4656d96866b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:53.655146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a40151-e56e-440a-adc8-8986b35eaaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:53.655963 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a40151-e56e-440a-adc8-8986b35eaaf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:53.656325 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a40151-e56e-440a-adc8-8986b35eaaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:53.656674 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a40151-e56e-440a-adc8-8986b35eaaf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:53.657493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a40151-e56e-440a-adc8-8986b35eaaf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:53.660243 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a40151-e56e-440a-adc8-8986b35eaaf5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:53.687617 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1131/2281] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:14:53.689774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8dc2d0-27fb-411d-8ed7-71e929a2b49b admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:53.701900 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8dc2d0-27fb-411d-8ed7-71e929a2b49b admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:53.708576 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4aae4-8d50-4a86-97b6-250fd7849462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:53.709261 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4aae4-8d50-4a86-97b6-250fd7849462 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:53.709641 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4aae4-8d50-4a86-97b6-250fd7849462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:53.709947 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4aae4-8d50-4a86-97b6-250fd7849462 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:53.710856 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4aae4-8d50-4a86-97b6-250fd7849462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:53.713099 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4aae4-8d50-4a86-97b6-250fd7849462 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:53.730804 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1151/2282] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:53.732193 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a40151-e56e-440a-adc8-8986b35eaaf5 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:53.755218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a40151-e56e-440a-adc8-8986b35eaaf5 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:53.763382 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7ef89-a239-4ed4-b94a-509aed4bd76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:53.764552 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7ef89-a239-4ed4-b94a-509aed4bd76d admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:53.764951 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7ef89-a239-4ed4-b94a-509aed4bd76d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:53.766395 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7ef89-a239-4ed4-b94a-509aed4bd76d admin admin] PATH_INFO: `/v3/projects/0bdbff6fbeb8492cb7b099dc3937dcf2/users/a63b690758a745bb95f2ec1468d57355/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:53.795141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7ef89-a239-4ed4-b94a-509aed4bd76d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bdbff6fbeb8492cb7b099dc3937dcf2, user_id=a63b690758a745bb95f2ec1468d57355, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:53.798416 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7ef89-a239-4ed4-b94a-509aed4bd76d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:53.821963 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1132/2283] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:14:53 2022] PUT /identity/v3/projects/0bdbff6fbeb8492cb7b099dc3937dcf2/users/a63b690758a745bb95f2ec1468d57355/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:53.845167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be942b33-5e70-407a-aaf3-c36366d76249 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:53.845965 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be942b33-5e70-407a-aaf3-c36366d76249 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:53.846530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be942b33-5e70-407a-aaf3-c36366d76249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:53.911369 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-be942b33-5e70-407a-aaf3-c36366d76249 None None] MFA Rules not processed for user `a63b690758a745bb95f2ec1468d57355`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:53.913500 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-be942b33-5e70-407a-aaf3-c36366d76249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:53.991508 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1152/2284] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:14:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:53.996574 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7ef89-a239-4ed4-b94a-509aed4bd76d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:54.078772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7ef89-a239-4ed4-b94a-509aed4bd76d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:54.084159 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda66d5-2406-4650-a3e3-2bcdd861d1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:54.085188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda66d5-2406-4650-a3e3-2bcdd861d1b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:54.085692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda66d5-2406-4650-a3e3-2bcdd861d1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:54.086650 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda66d5-2406-4650-a3e3-2bcdd861d1b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:54.088049 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcda66d5-2406-4650-a3e3-2bcdd861d1b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:54.089558 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcda66d5-2406-4650-a3e3-2bcdd861d1b1 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:54.125450 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1133/2285] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:14:53 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:14:54.129508 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be942b33-5e70-407a-aaf3-c36366d76249 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:54.141756 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be942b33-5e70-407a-aaf3-c36366d76249 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:54.235362 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2322241-df36-4693-8df0-02aa06bec27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:54.236436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2322241-df36-4693-8df0-02aa06bec27c admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:54.236887 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2322241-df36-4693-8df0-02aa06bec27c admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:54.237783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2322241-df36-4693-8df0-02aa06bec27c admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:54.238518 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2322241-df36-4693-8df0-02aa06bec27c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:54.241081 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2322241-df36-4693-8df0-02aa06bec27c admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:54.274541 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1153/2286] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:14:54 2022] POST /identity/v3/users => generated 299 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:14:54.275193 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda66d5-2406-4650-a3e3-2bcdd861d1b1 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:54.290156 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda66d5-2406-4650-a3e3-2bcdd861d1b1 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:54.297034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788d9ae-a203-41ce-bd06-bf2ac9683d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:54.298150 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788d9ae-a203-41ce-bd06-bf2ac9683d40 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:54.298547 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788d9ae-a203-41ce-bd06-bf2ac9683d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:54.299333 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788d9ae-a203-41ce-bd06-bf2ac9683d40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:54.299951 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b788d9ae-a203-41ce-bd06-bf2ac9683d40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:54.301947 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b788d9ae-a203-41ce-bd06-bf2ac9683d40 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:54.324157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1134/2287] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:54.326135 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2322241-df36-4693-8df0-02aa06bec27c admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:54.353622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2322241-df36-4693-8df0-02aa06bec27c admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:54.361677 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe7e0a-3afe-45c9-b150-2a413836c196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:54.363422 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe7e0a-3afe-45c9-b150-2a413836c196 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:54.363935 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe7e0a-3afe-45c9-b150-2a413836c196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:54.364425 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe7e0a-3afe-45c9-b150-2a413836c196 admin admin] PATH_INFO: `/v3/projects/93e66156ab0342abad3f0bbf09de4d29/users/ade8a58cd2534d65acfe93b0e6b23a15/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:54.420705 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fe7e0a-3afe-45c9-b150-2a413836c196 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93e66156ab0342abad3f0bbf09de4d29, user_id=ade8a58cd2534d65acfe93b0e6b23a15, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:54.434000 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fe7e0a-3afe-45c9-b150-2a413836c196 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:54.480313 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1154/2288] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:14:54 2022] PUT /identity/v3/projects/93e66156ab0342abad3f0bbf09de4d29/users/ade8a58cd2534d65acfe93b0e6b23a15/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:14:54.485282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788d9ae-a203-41ce-bd06-bf2ac9683d40 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:54.518685 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788d9ae-a203-41ce-bd06-bf2ac9683d40 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:54.733075 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6834c59a-f105-423f-9c7c-c88cfd4a7477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:54.735545 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834c59a-f105-423f-9c7c-c88cfd4a7477 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:54.737254 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834c59a-f105-423f-9c7c-c88cfd4a7477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:54.738911 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834c59a-f105-423f-9c7c-c88cfd4a7477 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:54.741086 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6834c59a-f105-423f-9c7c-c88cfd4a7477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:54.750478 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6834c59a-f105-423f-9c7c-c88cfd4a7477 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:54.825129 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe7e0a-3afe-45c9-b150-2a413836c196 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:54.826013 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1135/2289] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:54.869332 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe7e0a-3afe-45c9-b150-2a413836c196 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:54.896963 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1400ab03-31fd-419b-b07e-eaf2d3d805ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:54.903064 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400ab03-31fd-419b-b07e-eaf2d3d805ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:54.918852 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400ab03-31fd-419b-b07e-eaf2d3d805ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:54.919379 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400ab03-31fd-419b-b07e-eaf2d3d805ab admin admin] PATH_INFO: `/v3/projects/93e66156ab0342abad3f0bbf09de4d29/users/ade8a58cd2534d65acfe93b0e6b23a15/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:54.964837 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1400ab03-31fd-419b-b07e-eaf2d3d805ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=93e66156ab0342abad3f0bbf09de4d29, user_id=ade8a58cd2534d65acfe93b0e6b23a15, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:55.008507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1400ab03-31fd-419b-b07e-eaf2d3d805ab admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:55.038213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:55.070363 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1155/2290] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:14:54 2022] PUT /identity/v3/projects/93e66156ab0342abad3f0bbf09de4d29/users/ade8a58cd2534d65acfe93b0e6b23a15/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:55.073509 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899b1ee-4e3b-4f1d-a846-6dc4873621cb None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:55.073940 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899b1ee-4e3b-4f1d-a846-6dc4873621cb None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:55.074364 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899b1ee-4e3b-4f1d-a846-6dc4873621cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:55.121617 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:55.131563 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d899b1ee-4e3b-4f1d-a846-6dc4873621cb None None] MFA Rules not processed for user `ade8a58cd2534d65acfe93b0e6b23a15`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:55.135029 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d899b1ee-4e3b-4f1d-a846-6dc4873621cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:55.279831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1136/2291] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:14:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:55.298159 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1400ab03-31fd-419b-b07e-eaf2d3d805ab admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:55.303626 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-1400ab03-31fd-419b-b07e-eaf2d3d805ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:55.422661 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1400ab03-31fd-419b-b07e-eaf2d3d805ab admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:55.429447 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f97aa5-6e97-4d5e-814d-1619f574a30c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:55.430972 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f97aa5-6e97-4d5e-814d-1619f574a30c service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:55.431269 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f97aa5-6e97-4d5e-814d-1619f574a30c service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:55.431544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f97aa5-6e97-4d5e-814d-1619f574a30c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:55.445626 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f97aa5-6e97-4d5e-814d-1619f574a30c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:55.448750 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f97aa5-6e97-4d5e-814d-1619f574a30c service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:55.474136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1156/2292] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:14:55 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:14:56.967645 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d899b1ee-4e3b-4f1d-a846-6dc4873621cb None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:56.969848 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d899b1ee-4e3b-4f1d-a846-6dc4873621cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:57.016523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d899b1ee-4e3b-4f1d-a846-6dc4873621cb None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:57.043140 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feef974-b73b-4db0-b9d6-fa5656e003bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:57.044232 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feef974-b73b-4db0-b9d6-fa5656e003bd service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:57.044673 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feef974-b73b-4db0-b9d6-fa5656e003bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:57.045675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feef974-b73b-4db0-b9d6-fa5656e003bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:57.254714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feef974-b73b-4db0-b9d6-fa5656e003bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:57.256843 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feef974-b73b-4db0-b9d6-fa5656e003bd service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:57.302331 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1137/2293] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:14:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:14:58.419026 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a878d-7cc1-4cd2-b0dd-df243573fcd2 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:58.419026 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a878d-7cc1-4cd2-b0dd-df243573fcd2 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:58.419026 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a878d-7cc1-4cd2-b0dd-df243573fcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:58.536193 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-b98a878d-7cc1-4cd2-b0dd-df243573fcd2 None None] MFA Rules not processed for user `d5acd259bd7648aba86df057ea081655`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:58.538439 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-b98a878d-7cc1-4cd2-b0dd-df243573fcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:58.754416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1157/2294] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:14:58 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:58.806978 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feef974-b73b-4db0-b9d6-fa5656e003bd service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:58.809372 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1feef974-b73b-4db0-b9d6-fa5656e003bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:59.156199 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feef974-b73b-4db0-b9d6-fa5656e003bd service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:59.174639 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f7055-f527-4698-b611-268b369ba296 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f11beca9c5148eea4cc45f8dc89dbda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:59.176244 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f7055-f527-4698-b611-268b369ba296 service glance] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:59.177957 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f7055-f527-4698-b611-268b369ba296 service glance] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:59.179462 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f7055-f527-4698-b611-268b369ba296 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:59.192284 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7d4f7055-f527-4698-b611-268b369ba296 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:59.378373 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:59.507412 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f7055-f527-4698-b611-268b369ba296 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:59.512391 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f7055-f527-4698-b611-268b369ba296 service glance] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:59.533194 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1138/2295] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:14:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 727 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:14:59.884507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ec1ea-a367-48d2-a290-8018fcc55d25 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:59.887063 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ec1ea-a367-48d2-a290-8018fcc55d25 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:59.887799 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ec1ea-a367-48d2-a290-8018fcc55d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:00.114226 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-0a4ec1ea-a367-48d2-a290-8018fcc55d25 None None] MFA Rules not processed for user `2c2e0e19880f4b048faf034dd38c7365`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:00.117752 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-0a4ec1ea-a367-48d2-a290-8018fcc55d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:00.238272 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:00.306491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1158/2296] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 16:14:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 425 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:00.387742 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f7055-f527-4698-b611-268b369ba296 service glance] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:00.391263 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7d4f7055-f527-4698-b611-268b369ba296 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:00.730479 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f7055-f527-4698-b611-268b369ba296 service glance] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:00.738446 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bede1f0-02ab-4f7a-9421-c2de8884ea54 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0082e37ce54f49f7837601349508c89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:00.739394 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bede1f0-02ab-4f7a-9421-c2de8884ea54 service swift] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:00.739987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bede1f0-02ab-4f7a-9421-c2de8884ea54 service swift] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:00.740606 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bede1f0-02ab-4f7a-9421-c2de8884ea54 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:00.764629 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bede1f0-02ab-4f7a-9421-c2de8884ea54 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:00.769665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bede1f0-02ab-4f7a-9421-c2de8884ea54 service swift] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:00.798936 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1139/2297] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 16:15:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 16:15:04.164266 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ec1ea-a367-48d2-a290-8018fcc55d25 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:04.181401 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ec1ea-a367-48d2-a290-8018fcc55d25 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:04.185895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995db92-5492-45b9-a076-210fd0084a65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:04.186742 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995db92-5492-45b9-a076-210fd0084a65 service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:04.187447 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995db92-5492-45b9-a076-210fd0084a65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:04.188218 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995db92-5492-45b9-a076-210fd0084a65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:04.431286 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995db92-5492-45b9-a076-210fd0084a65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:04.435547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995db92-5492-45b9-a076-210fd0084a65 service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:04.475753 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1159/2298] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:15:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:15:09.027259 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bede1f0-02ab-4f7a-9421-c2de8884ea54 service swift] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:09.269755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bede1f0-02ab-4f7a-9421-c2de8884ea54 service swift] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:09.276874 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a55f4-9bed-4ea0-a6a4-71c3fda051f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:09.278113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a55f4-9bed-4ea0-a6a4-71c3fda051f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:09.278406 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a55f4-9bed-4ea0-a6a4-71c3fda051f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:09.278897 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a55f4-9bed-4ea0-a6a4-71c3fda051f9 admin admin] PATH_INFO: `/v3/users/882660f949ff4bce837eace270d48d53` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:09.283695 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a55f4-9bed-4ea0-a6a4-71c3fda051f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=882660f949ff4bce837eace270d48d53)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:09.288348 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a55f4-9bed-4ea0-a6a4-71c3fda051f9 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:09.330588 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a88de0-6952-4b85-831e-201390524be0 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:09.343181 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a88de0-6952-4b85-831e-201390524be0 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:09.343971 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a88de0-6952-4b85-831e-201390524be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:09.384309 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-c5a88de0-6952-4b85-831e-201390524be0 None None] MFA Rules not processed for user `515114f0a842463fb2d1265ee33db8eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:09.389994 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-c5a88de0-6952-4b85-831e-201390524be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:09.440791 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:09.449568 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-962a55f4-9bed-4ea0-a6a4-71c3fda051f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '882660f949ff4bce837eace270d48d53'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:09.484568 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-962a55f4-9bed-4ea0-a6a4-71c3fda051f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '882660f949ff4bce837eace270d48d53'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:09.508468 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-962a55f4-9bed-4ea0-a6a4-71c3fda051f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '882660f949ff4bce837eace270d48d53'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:09.522097 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-962a55f4-9bed-4ea0-a6a4-71c3fda051f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '882660f949ff4bce837eace270d48d53'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:09.523728 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-962a55f4-9bed-4ea0-a6a4-71c3fda051f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '882660f949ff4bce837eace270d48d53'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:09.535285 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1140/2299] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:15:09 2022] DELETE /identity/v3/users/882660f949ff4bce837eace270d48d53 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:09.546072 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1dbcd-4876-4c1a-939c-f7fba53a3a8a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:09.546556 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1dbcd-4876-4c1a-939c-f7fba53a3a8a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:09.546826 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1dbcd-4876-4c1a-939c-f7fba53a3a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:09.566324 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1160/2300] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:15:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:09.626158 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-44d1dbcd-4876-4c1a-939c-f7fba53a3a8a None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:09.627495 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-44d1dbcd-4876-4c1a-939c-f7fba53a3a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:09.751758 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1141/2301] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:09.771356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a88de0-6952-4b85-831e-201390524be0 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:09.773790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-c5a88de0-6952-4b85-831e-201390524be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:09.910082 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a88de0-6952-4b85-831e-201390524be0 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:10.032592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:10.050503 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb9bdf-5af5-4ef0-bf97-a15914a1604e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:10.052074 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb9bdf-5af5-4ef0-bf97-a15914a1604e service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:10.052814 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb9bdf-5af5-4ef0-bf97-a15914a1604e service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:10.053834 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb9bdf-5af5-4ef0-bf97-a15914a1604e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:10.080000 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb9bdf-5af5-4ef0-bf97-a15914a1604e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:10.082336 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb9bdf-5af5-4ef0-bf97-a15914a1604e service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:10.106672 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1161/2302] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:15:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:15:10.729617 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1dbcd-4876-4c1a-939c-f7fba53a3a8a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:10.733549 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-44d1dbcd-4876-4c1a-939c-f7fba53a3a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:10.772722 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1dbcd-4876-4c1a-939c-f7fba53a3a8a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:10.838979 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b6a37-0c66-445b-bca2-d23c33ffde9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:10.839890 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b6a37-0c66-445b-bca2-d23c33ffde9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:10.840327 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b6a37-0c66-445b-bca2-d23c33ffde9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:10.840609 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b6a37-0c66-445b-bca2-d23c33ffde9a admin admin] PATH_INFO: `/v3/projects/0ccbf228e6dc4df19b0c488297f36ada` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:10.845543 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b6a37-0c66-445b-bca2-d23c33ffde9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ccbf228e6dc4df19b0c488297f36ada)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:10.850051 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b6a37-0c66-445b-bca2-d23c33ffde9a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:10.985826 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b77b6a37-0c66-445b-bca2-d23c33ffde9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ccbf228e6dc4df19b0c488297f36ada', 'request_id': 'req-b77b6a37-0c66-445b-bca2-d23c33ffde9a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:11.000785 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b77b6a37-0c66-445b-bca2-d23c33ffde9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ccbf228e6dc4df19b0c488297f36ada', 'request_id': 'req-b77b6a37-0c66-445b-bca2-d23c33ffde9a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:11.012754 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b77b6a37-0c66-445b-bca2-d23c33ffde9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ccbf228e6dc4df19b0c488297f36ada', 'request_id': 'req-b77b6a37-0c66-445b-bca2-d23c33ffde9a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:11.029824 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b77b6a37-0c66-445b-bca2-d23c33ffde9a admin admin] The token cache is being invalidate because project 0ccbf228e6dc4df19b0c488297f36ada was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:15:11.030520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b77b6a37-0c66-445b-bca2-d23c33ffde9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:11.031326 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b77b6a37-0c66-445b-bca2-d23c33ffde9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:11.033977 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1142/2303] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:15:10 2022] DELETE /identity/v3/projects/0ccbf228e6dc4df19b0c488297f36ada => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:11.039161 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb9bdf-5af5-4ef0-bf97-a15914a1604e service neutron] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:11.053672 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb9bdf-5af5-4ef0-bf97-a15914a1604e service neutron] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:11.191736 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950cd0cb-82ce-4e69-bcfc-aee2f79d31c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:11.195451 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950cd0cb-82ce-4e69-bcfc-aee2f79d31c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:11.196386 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950cd0cb-82ce-4e69-bcfc-aee2f79d31c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:11.196905 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950cd0cb-82ce-4e69-bcfc-aee2f79d31c6 admin admin] PATH_INFO: `/v3/users/a63b690758a745bb95f2ec1468d57355` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:11.198900 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950cd0cb-82ce-4e69-bcfc-aee2f79d31c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a63b690758a745bb95f2ec1468d57355)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:11.201867 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950cd0cb-82ce-4e69-bcfc-aee2f79d31c6 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:11.327188 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-950cd0cb-82ce-4e69-bcfc-aee2f79d31c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a63b690758a745bb95f2ec1468d57355'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:11.349580 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-950cd0cb-82ce-4e69-bcfc-aee2f79d31c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a63b690758a745bb95f2ec1468d57355'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:11.399231 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-950cd0cb-82ce-4e69-bcfc-aee2f79d31c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a63b690758a745bb95f2ec1468d57355'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:11.400365 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-950cd0cb-82ce-4e69-bcfc-aee2f79d31c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a63b690758a745bb95f2ec1468d57355'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:11.415719 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-950cd0cb-82ce-4e69-bcfc-aee2f79d31c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a63b690758a745bb95f2ec1468d57355'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:11.441127 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1162/2304] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:15:11 2022] DELETE /identity/v3/users/a63b690758a745bb95f2ec1468d57355 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:12.100355 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b6a37-0c66-445b-bca2-d23c33ffde9a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:12.102264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b77b6a37-0c66-445b-bca2-d23c33ffde9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:12.141501 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b6a37-0c66-445b-bca2-d23c33ffde9a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:12.272286 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:12.306767 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f24550-856b-4c84-be11-1926f14626d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:12.308515 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f24550-856b-4c84-be11-1926f14626d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:12.309112 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f24550-856b-4c84-be11-1926f14626d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:12.309613 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f24550-856b-4c84-be11-1926f14626d0 admin admin] PATH_INFO: `/v3/projects/0bdbff6fbeb8492cb7b099dc3937dcf2` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:12.316039 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f24550-856b-4c84-be11-1926f14626d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bdbff6fbeb8492cb7b099dc3937dcf2)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:12.320658 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f24550-856b-4c84-be11-1926f14626d0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:12.461280 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-16f24550-856b-4c84-be11-1926f14626d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bdbff6fbeb8492cb7b099dc3937dcf2', 'request_id': 'req-16f24550-856b-4c84-be11-1926f14626d0'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:12.477660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-16f24550-856b-4c84-be11-1926f14626d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bdbff6fbeb8492cb7b099dc3937dcf2', 'request_id': 'req-16f24550-856b-4c84-be11-1926f14626d0'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:12.492739 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-16f24550-856b-4c84-be11-1926f14626d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bdbff6fbeb8492cb7b099dc3937dcf2', 'request_id': 'req-16f24550-856b-4c84-be11-1926f14626d0'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:12.532384 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-16f24550-856b-4c84-be11-1926f14626d0 admin admin] The token cache is being invalidate because project 0bdbff6fbeb8492cb7b099dc3937dcf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:15:12.533247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-16f24550-856b-4c84-be11-1926f14626d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:12.534612 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-16f24550-856b-4c84-be11-1926f14626d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:12.548241 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1143/2305] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:15:12 2022] DELETE /identity/v3/projects/0bdbff6fbeb8492cb7b099dc3937dcf2 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:12.549875 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950cd0cb-82ce-4e69-bcfc-aee2f79d31c6 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:12.551673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-950cd0cb-82ce-4e69-bcfc-aee2f79d31c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:12.577444 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950cd0cb-82ce-4e69-bcfc-aee2f79d31c6 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:12.740156 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7127f8-fc5f-46ef-8219-ff5ac566792b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:12.742628 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7127f8-fc5f-46ef-8219-ff5ac566792b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:12.745206 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7127f8-fc5f-46ef-8219-ff5ac566792b admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:12.745737 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7127f8-fc5f-46ef-8219-ff5ac566792b admin admin] PATH_INFO: `/v3/users/ade8a58cd2534d65acfe93b0e6b23a15` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:12.747907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7127f8-fc5f-46ef-8219-ff5ac566792b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ade8a58cd2534d65acfe93b0e6b23a15)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:12.751452 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7127f8-fc5f-46ef-8219-ff5ac566792b admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:12.761332 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:12.873138 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6b7127f8-fc5f-46ef-8219-ff5ac566792b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ade8a58cd2534d65acfe93b0e6b23a15'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:12.884774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6b7127f8-fc5f-46ef-8219-ff5ac566792b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ade8a58cd2534d65acfe93b0e6b23a15'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:12.938595 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6b7127f8-fc5f-46ef-8219-ff5ac566792b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ade8a58cd2534d65acfe93b0e6b23a15'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:12.942783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6b7127f8-fc5f-46ef-8219-ff5ac566792b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ade8a58cd2534d65acfe93b0e6b23a15'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:12.975742 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6b7127f8-fc5f-46ef-8219-ff5ac566792b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ade8a58cd2534d65acfe93b0e6b23a15'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:13.010868 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1163/2306] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:15:12 2022] DELETE /identity/v3/users/ade8a58cd2534d65acfe93b0e6b23a15 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:13.901415 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f24550-856b-4c84-be11-1926f14626d0 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:13.904388 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-16f24550-856b-4c84-be11-1926f14626d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:13.954516 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f24550-856b-4c84-be11-1926f14626d0 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:14.093465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445d116-c770-4166-94bc-f3887c2c1500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:14.094657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445d116-c770-4166-94bc-f3887c2c1500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:14.095932 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445d116-c770-4166-94bc-f3887c2c1500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:14.095932 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445d116-c770-4166-94bc-f3887c2c1500 admin admin] PATH_INFO: `/v3/projects/93e66156ab0342abad3f0bbf09de4d29` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:14.099248 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445d116-c770-4166-94bc-f3887c2c1500 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93e66156ab0342abad3f0bbf09de4d29)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:14.107746 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445d116-c770-4166-94bc-f3887c2c1500 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:14.256808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0445d116-c770-4166-94bc-f3887c2c1500 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93e66156ab0342abad3f0bbf09de4d29', 'request_id': 'req-0445d116-c770-4166-94bc-f3887c2c1500'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:14.271073 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0445d116-c770-4166-94bc-f3887c2c1500 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93e66156ab0342abad3f0bbf09de4d29', 'request_id': 'req-0445d116-c770-4166-94bc-f3887c2c1500'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:14.286919 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0445d116-c770-4166-94bc-f3887c2c1500 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93e66156ab0342abad3f0bbf09de4d29', 'request_id': 'req-0445d116-c770-4166-94bc-f3887c2c1500'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:14.316107 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0445d116-c770-4166-94bc-f3887c2c1500 admin admin] The token cache is being invalidate because project 93e66156ab0342abad3f0bbf09de4d29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:15:14.316107 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0445d116-c770-4166-94bc-f3887c2c1500 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:14.316107 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0445d116-c770-4166-94bc-f3887c2c1500 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:14.321165 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1144/2307] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:15:13 2022] DELETE /identity/v3/projects/93e66156ab0342abad3f0bbf09de4d29 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:14.326438 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e9c49-5281-4057-aab8-5ec0ed9476ce None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:14.327238 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e9c49-5281-4057-aab8-5ec0ed9476ce None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:14.327548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e9c49-5281-4057-aab8-5ec0ed9476ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:14.366089 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-259e9c49-5281-4057-aab8-5ec0ed9476ce None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:14.368828 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-259e9c49-5281-4057-aab8-5ec0ed9476ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:14.525009 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1164/2308] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:15:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:14.529598 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a209ccf-def9-45cc-8fb8-74b7b48d0eb9 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:14.537453 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a209ccf-def9-45cc-8fb8-74b7b48d0eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:14.538261 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a209ccf-def9-45cc-8fb8-74b7b48d0eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:14.602714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:14.644544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8a209ccf-def9-45cc-8fb8-74b7b48d0eb9 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:14.646677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8a209ccf-def9-45cc-8fb8-74b7b48d0eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:14.695708 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1145/2309] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:15:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:14.700359 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e9c49-5281-4057-aab8-5ec0ed9476ce None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:14.721981 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e9c49-5281-4057-aab8-5ec0ed9476ce None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:14.747231 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3aecd7-ac55-4abf-bdb5-0f1db2c00265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:14.748963 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3aecd7-ac55-4abf-bdb5-0f1db2c00265 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:14.749678 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3aecd7-ac55-4abf-bdb5-0f1db2c00265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:14.750457 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3aecd7-ac55-4abf-bdb5-0f1db2c00265 admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:14.752333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3aecd7-ac55-4abf-bdb5-0f1db2c00265 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:15:14.753484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3aecd7-ac55-4abf-bdb5-0f1db2c00265 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:14.758856 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3aecd7-ac55-4abf-bdb5-0f1db2c00265 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:14.785018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1165/2310] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:15:14.796410 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a209ccf-def9-45cc-8fb8-74b7b48d0eb9 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:14.820258 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a209ccf-def9-45cc-8fb8-74b7b48d0eb9 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:14.829508 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade4597-8af0-4f08-a88a-e0d9fff7a9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:14.831117 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade4597-8af0-4f08-a88a-e0d9fff7a9cf admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:14.831585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade4597-8af0-4f08-a88a-e0d9fff7a9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:14.832114 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade4597-8af0-4f08-a88a-e0d9fff7a9cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:14.833528 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade4597-8af0-4f08-a88a-e0d9fff7a9cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:14.837074 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade4597-8af0-4f08-a88a-e0d9fff7a9cf admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:14.929141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1146/2311] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:15:14 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:15:14.943406 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3aecd7-ac55-4abf-bdb5-0f1db2c00265 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:14.983151 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3aecd7-ac55-4abf-bdb5-0f1db2c00265 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:15.143049 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24dc94-1b9e-4b47-bc72-e247519f2c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:15.144498 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24dc94-1b9e-4b47-bc72-e247519f2c87 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:15.145164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24dc94-1b9e-4b47-bc72-e247519f2c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:15.146025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24dc94-1b9e-4b47-bc72-e247519f2c87 admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:15.147379 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb24dc94-1b9e-4b47-bc72-e247519f2c87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:15.152192 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb24dc94-1b9e-4b47-bc72-e247519f2c87 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:15.193530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:15.209548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1166/2312] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:15:14 2022] POST /identity/v3/users => generated 287 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:15:15.212508 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade4597-8af0-4f08-a88a-e0d9fff7a9cf admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:15.228937 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade4597-8af0-4f08-a88a-e0d9fff7a9cf admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:15.235740 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1e4b8-36d3-483f-be8f-babd3151e920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:15.236447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1e4b8-36d3-483f-be8f-babd3151e920 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:15.236826 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1e4b8-36d3-483f-be8f-babd3151e920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:15.237180 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1e4b8-36d3-483f-be8f-babd3151e920 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:15.237986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1e4b8-36d3-483f-be8f-babd3151e920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:15.242938 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1e4b8-36d3-483f-be8f-babd3151e920 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:15.282441 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1147/2313] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:15:15.289517 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24dc94-1b9e-4b47-bc72-e247519f2c87 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:15.374517 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24dc94-1b9e-4b47-bc72-e247519f2c87 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:15.384550 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d4f55-cc40-47ca-a440-f4666ed8a72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:15.385574 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d4f55-cc40-47ca-a440-f4666ed8a72b admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:15.386170 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d4f55-cc40-47ca-a440-f4666ed8a72b admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:15.386547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d4f55-cc40-47ca-a440-f4666ed8a72b admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:15.387606 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d4f55-cc40-47ca-a440-f4666ed8a72b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:15.389681 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d4f55-cc40-47ca-a440-f4666ed8a72b admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:15.427533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1167/2314] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:15:15.433645 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1e4b8-36d3-483f-be8f-babd3151e920 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:15.458194 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1e4b8-36d3-483f-be8f-babd3151e920 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:15.470690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c1e35-3937-4c2a-b7c0-e400f3c4547e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:15.471961 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c1e35-3937-4c2a-b7c0-e400f3c4547e admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:15.472292 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c1e35-3937-4c2a-b7c0-e400f3c4547e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:15.472621 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c1e35-3937-4c2a-b7c0-e400f3c4547e admin admin] PATH_INFO: `/v3/projects/c3dbb721d03d4ade92c27f5b6eb1cda1/users/2f307fd4499140308098e29a1014ee7f/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:15.513200 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c1e35-3937-4c2a-b7c0-e400f3c4547e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3dbb721d03d4ade92c27f5b6eb1cda1, user_id=2f307fd4499140308098e29a1014ee7f, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:15.513200 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c1e35-3937-4c2a-b7c0-e400f3c4547e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:15.547253 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1148/2315] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:15:15 2022] PUT /identity/v3/projects/c3dbb721d03d4ade92c27f5b6eb1cda1/users/2f307fd4499140308098e29a1014ee7f/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:15.569615 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c86d3-c8fb-4800-a4ab-37d2a3696399 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:15.571130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c86d3-c8fb-4800-a4ab-37d2a3696399 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:15.571130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c86d3-c8fb-4800-a4ab-37d2a3696399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:15.667102 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-566c86d3-c8fb-4800-a4ab-37d2a3696399 None None] MFA Rules not processed for user `2f307fd4499140308098e29a1014ee7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:15.669773 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-566c86d3-c8fb-4800-a4ab-37d2a3696399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:15.897394 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1168/2316] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:15:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:15.921098 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c1e35-3937-4c2a-b7c0-e400f3c4547e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:15.932960 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-778c1e35-3937-4c2a-b7c0-e400f3c4547e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:16.063459 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c1e35-3937-4c2a-b7c0-e400f3c4547e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:16.070739 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e82b20-23c9-46b0-8198-95eef2f61f5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:16.071902 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e82b20-23c9-46b0-8198-95eef2f61f5d service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:16.072391 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e82b20-23c9-46b0-8198-95eef2f61f5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:16.073052 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e82b20-23c9-46b0-8198-95eef2f61f5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:16.095404 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e82b20-23c9-46b0-8198-95eef2f61f5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:16.098226 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e82b20-23c9-46b0-8198-95eef2f61f5d service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:16.129264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1149/2317] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:15:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:15:16.158375 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c86d3-c8fb-4800-a4ab-37d2a3696399 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:16.325105 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c86d3-c8fb-4800-a4ab-37d2a3696399 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:16.337808 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428174d4-b471-4378-a2b2-9eae06682ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:16.339252 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428174d4-b471-4378-a2b2-9eae06682ce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:16.339543 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428174d4-b471-4378-a2b2-9eae06682ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:16.339832 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428174d4-b471-4378-a2b2-9eae06682ce1 admin admin] PATH_INFO: `/v3/users/2f307fd4499140308098e29a1014ee7f` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:16.341631 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428174d4-b471-4378-a2b2-9eae06682ce1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f307fd4499140308098e29a1014ee7f)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:16.343683 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428174d4-b471-4378-a2b2-9eae06682ce1 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:16.464433 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-428174d4-b471-4378-a2b2-9eae06682ce1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f307fd4499140308098e29a1014ee7f'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:16.491077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-428174d4-b471-4378-a2b2-9eae06682ce1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f307fd4499140308098e29a1014ee7f'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:16.529890 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-428174d4-b471-4378-a2b2-9eae06682ce1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f307fd4499140308098e29a1014ee7f'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:16.531292 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-428174d4-b471-4378-a2b2-9eae06682ce1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f307fd4499140308098e29a1014ee7f'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:16.552143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-428174d4-b471-4378-a2b2-9eae06682ce1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f307fd4499140308098e29a1014ee7f'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:16.563923 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:16.571420 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1169/2318] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:15:16 2022] DELETE /identity/v3/users/2f307fd4499140308098e29a1014ee7f => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:16.582667 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf4746e-d622-4b05-bbcb-d0cf8f0b2645 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:16.583589 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf4746e-d622-4b05-bbcb-d0cf8f0b2645 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:16.584521 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf4746e-d622-4b05-bbcb-d0cf8f0b2645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:16.627074 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-5bf4746e-d622-4b05-bbcb-d0cf8f0b2645 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:16.630085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5bf4746e-d622-4b05-bbcb-d0cf8f0b2645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:16.822772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1150/2319] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:15:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:16.838654 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428174d4-b471-4378-a2b2-9eae06682ce1 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:16.842763 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-428174d4-b471-4378-a2b2-9eae06682ce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:16.983737 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428174d4-b471-4378-a2b2-9eae06682ce1 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:16.999473 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b87fa-bc15-4cbb-9849-d15fe103d299 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:17.000163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b87fa-bc15-4cbb-9849-d15fe103d299 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:17.001361 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b87fa-bc15-4cbb-9849-d15fe103d299 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:17.001361 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b87fa-bc15-4cbb-9849-d15fe103d299 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:17.019655 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b87fa-bc15-4cbb-9849-d15fe103d299 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:17.022564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b87fa-bc15-4cbb-9849-d15fe103d299 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:17.040710 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1170/2320] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:15:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:15:17.818171 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf4746e-d622-4b05-bbcb-d0cf8f0b2645 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:17.821368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5bf4746e-d622-4b05-bbcb-d0cf8f0b2645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:17.859553 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf4746e-d622-4b05-bbcb-d0cf8f0b2645 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:17.879083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9542d-8e9b-4f17-86fe-e3916e2ee3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:17.881444 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9542d-8e9b-4f17-86fe-e3916e2ee3e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:17.881872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9542d-8e9b-4f17-86fe-e3916e2ee3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:17.892043 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9542d-8e9b-4f17-86fe-e3916e2ee3e3 admin admin] PATH_INFO: `/v3/projects/c3dbb721d03d4ade92c27f5b6eb1cda1` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:17.902181 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa9542d-8e9b-4f17-86fe-e3916e2ee3e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3dbb721d03d4ade92c27f5b6eb1cda1)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:17.908690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa9542d-8e9b-4f17-86fe-e3916e2ee3e3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:18.005175 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:18.022528 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ffa9542d-8e9b-4f17-86fe-e3916e2ee3e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3dbb721d03d4ade92c27f5b6eb1cda1', 'request_id': 'req-ffa9542d-8e9b-4f17-86fe-e3916e2ee3e3'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:18.039060 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ffa9542d-8e9b-4f17-86fe-e3916e2ee3e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3dbb721d03d4ade92c27f5b6eb1cda1', 'request_id': 'req-ffa9542d-8e9b-4f17-86fe-e3916e2ee3e3'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:18.063666 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ffa9542d-8e9b-4f17-86fe-e3916e2ee3e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3dbb721d03d4ade92c27f5b6eb1cda1', 'request_id': 'req-ffa9542d-8e9b-4f17-86fe-e3916e2ee3e3'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:18.085026 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ffa9542d-8e9b-4f17-86fe-e3916e2ee3e3 admin admin] The token cache is being invalidate because project c3dbb721d03d4ade92c27f5b6eb1cda1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:15:18.085969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ffa9542d-8e9b-4f17-86fe-e3916e2ee3e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:18.087005 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ffa9542d-8e9b-4f17-86fe-e3916e2ee3e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:18.091063 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1151/2321] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:15:17 2022] DELETE /identity/v3/projects/c3dbb721d03d4ade92c27f5b6eb1cda1 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:18.094841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650879a-34fc-45b8-8b5e-d82dbfeccaad None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:18.095633 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650879a-34fc-45b8-8b5e-d82dbfeccaad None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:18.095633 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650879a-34fc-45b8-8b5e-d82dbfeccaad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:18.116982 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-9650879a-34fc-45b8-8b5e-d82dbfeccaad None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:18.118501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-9650879a-34fc-45b8-8b5e-d82dbfeccaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:18.225505 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1171/2322] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:18.236025 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e727e-e4a3-446f-81b3-73dec8abb0c8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:18.237660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e727e-e4a3-446f-81b3-73dec8abb0c8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:18.238409 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e727e-e4a3-446f-81b3-73dec8abb0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:18.288054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-5b9e727e-e4a3-446f-81b3-73dec8abb0c8 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:18.297717 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5b9e727e-e4a3-446f-81b3-73dec8abb0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:18.316922 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1152/2323] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:18.324891 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650879a-34fc-45b8-8b5e-d82dbfeccaad None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:18.343579 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650879a-34fc-45b8-8b5e-d82dbfeccaad None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:18.350045 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d52ba5a-b009-47c3-9bec-99bb438c0bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:18.351255 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d52ba5a-b009-47c3-9bec-99bb438c0bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:18.351971 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d52ba5a-b009-47c3-9bec-99bb438c0bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:18.352267 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d52ba5a-b009-47c3-9bec-99bb438c0bb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:18.353015 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d52ba5a-b009-47c3-9bec-99bb438c0bb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:15:18.353344 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d52ba5a-b009-47c3-9bec-99bb438c0bb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:18.355170 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d52ba5a-b009-47c3-9bec-99bb438c0bb3 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:18.378030 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1172/2324] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:15:18.378744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e727e-e4a3-446f-81b3-73dec8abb0c8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:18.390321 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e727e-e4a3-446f-81b3-73dec8abb0c8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:18.395950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70658ba0-7c83-4b54-ac45-15c302df4d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:18.396714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70658ba0-7c83-4b54-ac45-15c302df4d36 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:18.397099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70658ba0-7c83-4b54-ac45-15c302df4d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:18.397387 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70658ba0-7c83-4b54-ac45-15c302df4d36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:18.398311 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70658ba0-7c83-4b54-ac45-15c302df4d36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:18.400592 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70658ba0-7c83-4b54-ac45-15c302df4d36 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:18.461872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1153/2325] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:15:18 2022] POST /identity/v3/projects => generated 375 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:15:18.468934 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d52ba5a-b009-47c3-9bec-99bb438c0bb3 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:18.490492 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d52ba5a-b009-47c3-9bec-99bb438c0bb3 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:18.625441 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d85455-a2c1-400f-bdbd-9f53eef76f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:18.626162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d85455-a2c1-400f-bdbd-9f53eef76f19 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:18.626453 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d85455-a2c1-400f-bdbd-9f53eef76f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:18.626729 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d85455-a2c1-400f-bdbd-9f53eef76f19 admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:18.627504 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d85455-a2c1-400f-bdbd-9f53eef76f19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:18.635634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d85455-a2c1-400f-bdbd-9f53eef76f19 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:18.699025 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70658ba0-7c83-4b54-ac45-15c302df4d36 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:18.713471 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1173/2326] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:15:18 2022] POST /identity/v3/users => generated 288 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:15:18.736956 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70658ba0-7c83-4b54-ac45-15c302df4d36 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:18.784047 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b6f84-a483-4f34-9c93-30a133dbe61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:18.785450 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b6f84-a483-4f34-9c93-30a133dbe61b admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:18.786251 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b6f84-a483-4f34-9c93-30a133dbe61b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:18.787005 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b6f84-a483-4f34-9c93-30a133dbe61b admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:18.797161 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b6f84-a483-4f34-9c93-30a133dbe61b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:18.800283 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b6f84-a483-4f34-9c93-30a133dbe61b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:18.823632 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1154/2327] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:15:18.829850 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d85455-a2c1-400f-bdbd-9f53eef76f19 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:18.854787 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d85455-a2c1-400f-bdbd-9f53eef76f19 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:18.864384 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0e316-aa81-49f7-b603-d4cdbe569bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:18.865525 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0e316-aa81-49f7-b603-d4cdbe569bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:18.866077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0e316-aa81-49f7-b603-d4cdbe569bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:18.867800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0e316-aa81-49f7-b603-d4cdbe569bc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:18.869225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0e316-aa81-49f7-b603-d4cdbe569bc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:18.872495 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0e316-aa81-49f7-b603-d4cdbe569bc9 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:18.894934 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1174/2328] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:15:18.901647 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b6f84-a483-4f34-9c93-30a133dbe61b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:18.917767 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b6f84-a483-4f34-9c93-30a133dbe61b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:18.924079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18cfa5-44b3-439d-972a-94e000af6adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:18.925111 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18cfa5-44b3-439d-972a-94e000af6adb admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:18.925463 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18cfa5-44b3-439d-972a-94e000af6adb admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:18.925811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18cfa5-44b3-439d-972a-94e000af6adb admin admin] PATH_INFO: `/v3/projects/b1dbcbccf831411989b881787d9f04c5/users/1b786df5c7c744a4907f8908ce71f65c/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:18.948311 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a18cfa5-44b3-439d-972a-94e000af6adb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1dbcbccf831411989b881787d9f04c5, user_id=1b786df5c7c744a4907f8908ce71f65c, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:18.951206 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a18cfa5-44b3-439d-972a-94e000af6adb admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:18.982578 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1155/2329] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:15:18 2022] PUT /identity/v3/projects/b1dbcbccf831411989b881787d9f04c5/users/1b786df5c7c744a4907f8908ce71f65c/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:18.999661 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe90b38-5986-4f3c-9293-85fa1d46be10 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:19.000583 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe90b38-5986-4f3c-9293-85fa1d46be10 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:19.001259 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe90b38-5986-4f3c-9293-85fa1d46be10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:19.044506 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:19.079246 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-9fe90b38-5986-4f3c-9293-85fa1d46be10 None None] MFA Rules not processed for user `1b786df5c7c744a4907f8908ce71f65c`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:19.081045 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-9fe90b38-5986-4f3c-9293-85fa1d46be10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:19.308975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1175/2330] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:15:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:19.323778 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18cfa5-44b3-439d-972a-94e000af6adb admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:19.327185 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0a18cfa5-44b3-439d-972a-94e000af6adb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:19.474801 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18cfa5-44b3-439d-972a-94e000af6adb admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:19.481777 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a72a9-2241-4aa9-8cf3-387b8e887ff0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:19.483118 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a72a9-2241-4aa9-8cf3-387b8e887ff0 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:19.484065 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a72a9-2241-4aa9-8cf3-387b8e887ff0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:19.484569 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a72a9-2241-4aa9-8cf3-387b8e887ff0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:19.499534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a72a9-2241-4aa9-8cf3-387b8e887ff0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:19.501026 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a72a9-2241-4aa9-8cf3-387b8e887ff0 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:19.518734 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1156/2331] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:15:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:15:22.209252 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cf8bc-5aed-4bab-93fc-590c8a9d2eb8 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:22.210210 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cf8bc-5aed-4bab-93fc-590c8a9d2eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:22.210210 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cf8bc-5aed-4bab-93fc-590c8a9d2eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:22.251167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-4e5cf8bc-5aed-4bab-93fc-590c8a9d2eb8 None None] MFA Rules not processed for user `c0da50b34820422193be2b06e0da2837`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:22.252617 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-4e5cf8bc-5aed-4bab-93fc-590c8a9d2eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:22.410057 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1176/2332] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:15:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:29.510650 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=63,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:32.392964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=54 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:34.106404 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da8bfe-fd37-4bcd-a181-a79cb0b0d171 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:34.109529 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da8bfe-fd37-4bcd-a181-a79cb0b0d171 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:34.109529 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da8bfe-fd37-4bcd-a181-a79cb0b0d171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:34.138190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-34da8bfe-fd37-4bcd-a181-a79cb0b0d171 None None] MFA Rules not processed for user `2c2e0e19880f4b048faf034dd38c7365`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:34.141538 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-34da8bfe-fd37-4bcd-a181-a79cb0b0d171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:34.253309 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1157/2333] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 16:15:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:34.263937 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5cf8bc-5aed-4bab-93fc-590c8a9d2eb8 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:34.265520 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-4e5cf8bc-5aed-4bab-93fc-590c8a9d2eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:34.413727 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5cf8bc-5aed-4bab-93fc-590c8a9d2eb8 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:34.425711 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c24fcf-6c3e-43ed-be4f-0cc58f4180c4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0082e37ce54f49f7837601349508c89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:34.427123 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c24fcf-6c3e-43ed-be4f-0cc58f4180c4 service swift] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:34.427790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c24fcf-6c3e-43ed-be4f-0cc58f4180c4 service swift] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:34.429229 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c24fcf-6c3e-43ed-be4f-0cc58f4180c4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:34.464014 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c24fcf-6c3e-43ed-be4f-0cc58f4180c4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:34.479850 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c24fcf-6c3e-43ed-be4f-0cc58f4180c4 service swift] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:34.499740 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1177/2334] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 16:15:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:15:35.914750 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313dcb6-e6d5-4a47-9615-352e6d860d22 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:35.915901 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313dcb6-e6d5-4a47-9615-352e6d860d22 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:35.917038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313dcb6-e6d5-4a47-9615-352e6d860d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:35.975078 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9313dcb6-e6d5-4a47-9615-352e6d860d22 None None] MFA Rules not processed for user `2c2e0e19880f4b048faf034dd38c7365`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:35.977062 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9313dcb6-e6d5-4a47-9615-352e6d860d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:35.988574 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1158/2335] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 16:15:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:15:36.025180 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c24fcf-6c3e-43ed-be4f-0cc58f4180c4 service swift] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:36.044909 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c24fcf-6c3e-43ed-be4f-0cc58f4180c4 service swift] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:36.059097 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd8c2a-7f67-49bd-801f-ebbd3d53a391 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0082e37ce54f49f7837601349508c89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:36.060105 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd8c2a-7f67-49bd-801f-ebbd3d53a391 service swift] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:36.060590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd8c2a-7f67-49bd-801f-ebbd3d53a391 service swift] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:36.061106 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd8c2a-7f67-49bd-801f-ebbd3d53a391 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:36.088148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd8c2a-7f67-49bd-801f-ebbd3d53a391 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:36.092545 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd8c2a-7f67-49bd-801f-ebbd3d53a391 service swift] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:36.111283 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1178/2336] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 16:15:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 16:15:45.958333 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=28 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:46.104046 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=38 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:46.839899 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313dcb6-e6d5-4a47-9615-352e6d860d22 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:46.842786 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9313dcb6-e6d5-4a47-9615-352e6d860d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:46.971351 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313dcb6-e6d5-4a47-9615-352e6d860d22 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:46.984102 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90912c55-0555-45d0-b0c8-f1b0b3d98e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:46.985903 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90912c55-0555-45d0-b0c8-f1b0b3d98e3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:46.986595 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90912c55-0555-45d0-b0c8-f1b0b3d98e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:46.987235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90912c55-0555-45d0-b0c8-f1b0b3d98e3b admin admin] PATH_INFO: `/v3/users/e0d2811ee47a427ab91069efbbc5806c` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:46.990076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90912c55-0555-45d0-b0c8-f1b0b3d98e3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0d2811ee47a427ab91069efbbc5806c)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:46.993362 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90912c55-0555-45d0-b0c8-f1b0b3d98e3b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:47.087433 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-90912c55-0555-45d0-b0c8-f1b0b3d98e3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0d2811ee47a427ab91069efbbc5806c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:47.143610 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-90912c55-0555-45d0-b0c8-f1b0b3d98e3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0d2811ee47a427ab91069efbbc5806c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:47.172955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-90912c55-0555-45d0-b0c8-f1b0b3d98e3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0d2811ee47a427ab91069efbbc5806c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:47.198726 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-90912c55-0555-45d0-b0c8-f1b0b3d98e3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0d2811ee47a427ab91069efbbc5806c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:47.201901 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-90912c55-0555-45d0-b0c8-f1b0b3d98e3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0d2811ee47a427ab91069efbbc5806c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:47.222251 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1159/2337] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:15:46 2022] DELETE /identity/v3/users/e0d2811ee47a427ab91069efbbc5806c => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:47.229500 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c33e0-d5c1-49e9-9c6b-3674f2a60545 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:47.230973 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c33e0-d5c1-49e9-9c6b-3674f2a60545 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:47.231663 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c33e0-d5c1-49e9-9c6b-3674f2a60545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:47.295300 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-2f0c33e0-d5c1-49e9-9c6b-3674f2a60545 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:47.298155 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-2f0c33e0-d5c1-49e9-9c6b-3674f2a60545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:47.399240 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1179/2338] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:15:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:47.410024 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90912c55-0555-45d0-b0c8-f1b0b3d98e3b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:47.414228 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-90912c55-0555-45d0-b0c8-f1b0b3d98e3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:47.733707 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90912c55-0555-45d0-b0c8-f1b0b3d98e3b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:47.769342 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce20ce-b829-4575-936e-99a212c2c057 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:47.771673 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce20ce-b829-4575-936e-99a212c2c057 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:47.775577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce20ce-b829-4575-936e-99a212c2c057 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:47.776433 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce20ce-b829-4575-936e-99a212c2c057 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:47.833864 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ce20ce-b829-4575-936e-99a212c2c057 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:47.837402 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ce20ce-b829-4575-936e-99a212c2c057 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:47.882856 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1160/2339] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:15:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:15:50.089301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c33e0-d5c1-49e9-9c6b-3674f2a60545 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:50.092546 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-2f0c33e0-d5c1-49e9-9c6b-3674f2a60545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:50.138736 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c33e0-d5c1-49e9-9c6b-3674f2a60545 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:50.153443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22534452-7582-4dda-a943-14eee6965430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:50.155020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22534452-7582-4dda-a943-14eee6965430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:50.155756 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22534452-7582-4dda-a943-14eee6965430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:50.156356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22534452-7582-4dda-a943-14eee6965430 admin admin] PATH_INFO: `/v3/projects/8ee8a35d0ab2411488227c7e2084be97` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:50.161176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22534452-7582-4dda-a943-14eee6965430 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ee8a35d0ab2411488227c7e2084be97)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:50.165082 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22534452-7582-4dda-a943-14eee6965430 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:50.337481 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-22534452-7582-4dda-a943-14eee6965430 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ee8a35d0ab2411488227c7e2084be97', 'request_id': 'req-22534452-7582-4dda-a943-14eee6965430'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:50.345522 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-22534452-7582-4dda-a943-14eee6965430 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ee8a35d0ab2411488227c7e2084be97', 'request_id': 'req-22534452-7582-4dda-a943-14eee6965430'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:50.355507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-22534452-7582-4dda-a943-14eee6965430 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ee8a35d0ab2411488227c7e2084be97', 'request_id': 'req-22534452-7582-4dda-a943-14eee6965430'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:50.381400 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-22534452-7582-4dda-a943-14eee6965430 admin admin] The token cache is being invalidate because project 8ee8a35d0ab2411488227c7e2084be97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:15:50.383534 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-22534452-7582-4dda-a943-14eee6965430 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:50.386545 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-22534452-7582-4dda-a943-14eee6965430 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:50.391327 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1180/2340] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:15:50 2022] DELETE /identity/v3/projects/8ee8a35d0ab2411488227c7e2084be97 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:50.906214 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce20ce-b829-4575-936e-99a212c2c057 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:50.908011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d6ce20ce-b829-4575-936e-99a212c2c057 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:50.932146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce20ce-b829-4575-936e-99a212c2c057 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:51.011341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:51.144347 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef3b5c-d545-41a0-9ccb-15d5258a582a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:51.145559 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef3b5c-d545-41a0-9ccb-15d5258a582a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:51.146203 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef3b5c-d545-41a0-9ccb-15d5258a582a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:51.147247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef3b5c-d545-41a0-9ccb-15d5258a582a admin admin] PATH_INFO: `/v3/users/1b786df5c7c744a4907f8908ce71f65c` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:51.149070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef3b5c-d545-41a0-9ccb-15d5258a582a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b786df5c7c744a4907f8908ce71f65c)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:51.156914 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef3b5c-d545-41a0-9ccb-15d5258a582a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:51.318581 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-21ef3b5c-d545-41a0-9ccb-15d5258a582a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b786df5c7c744a4907f8908ce71f65c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:51.348445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-21ef3b5c-d545-41a0-9ccb-15d5258a582a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b786df5c7c744a4907f8908ce71f65c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:51.361743 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-21ef3b5c-d545-41a0-9ccb-15d5258a582a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b786df5c7c744a4907f8908ce71f65c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:51.374782 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-21ef3b5c-d545-41a0-9ccb-15d5258a582a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b786df5c7c744a4907f8908ce71f65c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:51.375779 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-21ef3b5c-d545-41a0-9ccb-15d5258a582a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b786df5c7c744a4907f8908ce71f65c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:51.392185 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1161/2341] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:15:50 2022] DELETE /identity/v3/users/1b786df5c7c744a4907f8908ce71f65c => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:15:51.397174 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc8c56-2f58-4384-b25f-7fc2cd34a5da None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:51.398577 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc8c56-2f58-4384-b25f-7fc2cd34a5da None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:51.398577 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc8c56-2f58-4384-b25f-7fc2cd34a5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:51.443422 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-aafc8c56-2f58-4384-b25f-7fc2cd34a5da None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:51.448812 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-aafc8c56-2f58-4384-b25f-7fc2cd34a5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:51.551939 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1181/2342] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:51.569697 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef3b5c-d545-41a0-9ccb-15d5258a582a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:51.572272 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-21ef3b5c-d545-41a0-9ccb-15d5258a582a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:51.758938 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef3b5c-d545-41a0-9ccb-15d5258a582a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:51.763809 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d798a-1602-4e63-af7f-c63bb390ae71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:51.764957 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d798a-1602-4e63-af7f-c63bb390ae71 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:51.765521 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d798a-1602-4e63-af7f-c63bb390ae71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:51.766149 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d798a-1602-4e63-af7f-c63bb390ae71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:51.784302 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d798a-1602-4e63-af7f-c63bb390ae71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:51.806067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d798a-1602-4e63-af7f-c63bb390ae71 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:51.842142 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1162/2343] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:15:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:15:52.513056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc8c56-2f58-4384-b25f-7fc2cd34a5da None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:52.517464 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-aafc8c56-2f58-4384-b25f-7fc2cd34a5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:52.556605 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc8c56-2f58-4384-b25f-7fc2cd34a5da None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:52.583239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e953f24a-c5bf-4957-9935-89d489476ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:52.587446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953f24a-c5bf-4957-9935-89d489476ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:52.590473 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953f24a-c5bf-4957-9935-89d489476ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:52.591115 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953f24a-c5bf-4957-9935-89d489476ac5 admin admin] PATH_INFO: `/v3/projects/b1dbcbccf831411989b881787d9f04c5` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:52.595029 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e953f24a-c5bf-4957-9935-89d489476ac5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1dbcbccf831411989b881787d9f04c5)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:52.603698 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e953f24a-c5bf-4957-9935-89d489476ac5 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:52.712772 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:52.813693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-e953f24a-c5bf-4957-9935-89d489476ac5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1dbcbccf831411989b881787d9f04c5', 'request_id': 'req-e953f24a-c5bf-4957-9935-89d489476ac5'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:52.847727 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-e953f24a-c5bf-4957-9935-89d489476ac5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1dbcbccf831411989b881787d9f04c5', 'request_id': 'req-e953f24a-c5bf-4957-9935-89d489476ac5'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:52.897979 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-e953f24a-c5bf-4957-9935-89d489476ac5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1dbcbccf831411989b881787d9f04c5', 'request_id': 'req-e953f24a-c5bf-4957-9935-89d489476ac5'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:52.996399 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-e953f24a-c5bf-4957-9935-89d489476ac5 admin admin] The token cache is being invalidate because project b1dbcbccf831411989b881787d9f04c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:15:52.996399 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-e953f24a-c5bf-4957-9935-89d489476ac5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:52.997196 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-e953f24a-c5bf-4957-9935-89d489476ac5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:53.003459 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1182/2344] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:15:52 2022] DELETE /identity/v3/projects/b1dbcbccf831411989b881787d9f04c5 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:53.014756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17ac56-396b-4ba7-af1a-5f38cc26fdf2 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:53.020375 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17ac56-396b-4ba7-af1a-5f38cc26fdf2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:53.020919 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17ac56-396b-4ba7-af1a-5f38cc26fdf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:53.083071 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-fc17ac56-396b-4ba7-af1a-5f38cc26fdf2 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:53.085279 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-fc17ac56-396b-4ba7-af1a-5f38cc26fdf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:53.216411 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:53.278190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1163/2345] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:53.282396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b1a2b-3d14-4a63-9418-62dde51d1ad1 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:53.283050 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b1a2b-3d14-4a63-9418-62dde51d1ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:53.283849 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b1a2b-3d14-4a63-9418-62dde51d1ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:53.323946 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-3c6b1a2b-3d14-4a63-9418-62dde51d1ad1 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:53.327160 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-3c6b1a2b-3d14-4a63-9418-62dde51d1ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:53.346659 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1183/2346] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:53.353594 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc17ac56-396b-4ba7-af1a-5f38cc26fdf2 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:53.378237 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc17ac56-396b-4ba7-af1a-5f38cc26fdf2 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:53.385343 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da255df1-655c-4c0b-9d07-1958c21252df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:53.386661 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da255df1-655c-4c0b-9d07-1958c21252df admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:53.387232 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da255df1-655c-4c0b-9d07-1958c21252df admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:53.387805 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da255df1-655c-4c0b-9d07-1958c21252df admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:53.389342 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da255df1-655c-4c0b-9d07-1958c21252df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:15:53.389994 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da255df1-655c-4c0b-9d07-1958c21252df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:53.392704 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da255df1-655c-4c0b-9d07-1958c21252df admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:53.427434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1164/2347] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:15:53.436391 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b1a2b-3d14-4a63-9418-62dde51d1ad1 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:53.457103 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b1a2b-3d14-4a63-9418-62dde51d1ad1 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:53.469343 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7beeb9d-fc8c-4047-af86-cb2d95439fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:53.470349 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7beeb9d-fc8c-4047-af86-cb2d95439fec admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:53.471316 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7beeb9d-fc8c-4047-af86-cb2d95439fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:53.471533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7beeb9d-fc8c-4047-af86-cb2d95439fec admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:53.472824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7beeb9d-fc8c-4047-af86-cb2d95439fec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:53.476509 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7beeb9d-fc8c-4047-af86-cb2d95439fec admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:53.542718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1184/2348] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:15:53 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:15:53.548934 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da255df1-655c-4c0b-9d07-1958c21252df admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:53.575667 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da255df1-655c-4c0b-9d07-1958c21252df admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:53.804620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5e5db-6ff1-41ce-8363-d25c7141382a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:53.808755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5e5db-6ff1-41ce-8363-d25c7141382a admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:53.810665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5e5db-6ff1-41ce-8363-d25c7141382a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:53.814540 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5e5db-6ff1-41ce-8363-d25c7141382a admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:53.821831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5e5db-6ff1-41ce-8363-d25c7141382a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:53.836596 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5e5db-6ff1-41ce-8363-d25c7141382a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:53.947508 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1165/2349] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:15:53 2022] POST /identity/v3/users => generated 287 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:15:53.953694 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7beeb9d-fc8c-4047-af86-cb2d95439fec admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:53.976650 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7beeb9d-fc8c-4047-af86-cb2d95439fec admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:53.988962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408e1d1-2334-409c-8d55-39209a79a90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:53.992013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408e1d1-2334-409c-8d55-39209a79a90c admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:53.992918 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408e1d1-2334-409c-8d55-39209a79a90c admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:53.997111 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408e1d1-2334-409c-8d55-39209a79a90c admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:53.999239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0408e1d1-2334-409c-8d55-39209a79a90c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:54.008539 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0408e1d1-2334-409c-8d55-39209a79a90c admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:54.038339 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1185/2350] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:15:54.039786 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5e5db-6ff1-41ce-8363-d25c7141382a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:54.097051 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5e5db-6ff1-41ce-8363-d25c7141382a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:54.125962 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0da9fb-af86-43fe-b5df-93d21de46a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:54.127841 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0da9fb-af86-43fe-b5df-93d21de46a22 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:54.128533 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0da9fb-af86-43fe-b5df-93d21de46a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:54.129055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0da9fb-af86-43fe-b5df-93d21de46a22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:54.132180 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0da9fb-af86-43fe-b5df-93d21de46a22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:54.136542 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0da9fb-af86-43fe-b5df-93d21de46a22 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:54.167940 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1166/2351] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:15:54.173552 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408e1d1-2334-409c-8d55-39209a79a90c admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:54.205963 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408e1d1-2334-409c-8d55-39209a79a90c admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:54.229213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a799aa5-93a1-486e-8a3b-0713c1a4c63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:54.231328 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a799aa5-93a1-486e-8a3b-0713c1a4c63b admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:54.232118 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a799aa5-93a1-486e-8a3b-0713c1a4c63b admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:54.232853 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a799aa5-93a1-486e-8a3b-0713c1a4c63b admin admin] PATH_INFO: `/v3/projects/59639e2c7ad9463990169380a66e682e/users/4d5875a4081c4c61a5c0208278f8edc1/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:54.270475 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a799aa5-93a1-486e-8a3b-0713c1a4c63b admin admin] RBAC: Authorizing `identity:create_grant(project_id=59639e2c7ad9463990169380a66e682e, user_id=4d5875a4081c4c61a5c0208278f8edc1, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:54.275227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a799aa5-93a1-486e-8a3b-0713c1a4c63b admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:54.324762 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1186/2352] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:15:54 2022] PUT /identity/v3/projects/59639e2c7ad9463990169380a66e682e/users/4d5875a4081c4c61a5c0208278f8edc1/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:54.332209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b980f-0193-4d60-afc4-e234f6e9e1dd None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:54.332535 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b980f-0193-4d60-afc4-e234f6e9e1dd None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:54.332788 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b980f-0193-4d60-afc4-e234f6e9e1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:54.402794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-cb6b980f-0193-4d60-afc4-e234f6e9e1dd None None] MFA Rules not processed for user `4d5875a4081c4c61a5c0208278f8edc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:54.404465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-cb6b980f-0193-4d60-afc4-e234f6e9e1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:54.569985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1167/2353] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:15:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:54.585416 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a799aa5-93a1-486e-8a3b-0713c1a4c63b admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:54.588214 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-9a799aa5-93a1-486e-8a3b-0713c1a4c63b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:54.761249 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a799aa5-93a1-486e-8a3b-0713c1a4c63b admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:54.766994 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033f8b8-881d-43ef-8b96-543f20f6e61a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:54.767989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033f8b8-881d-43ef-8b96-543f20f6e61a service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:54.768419 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033f8b8-881d-43ef-8b96-543f20f6e61a service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:54.768904 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033f8b8-881d-43ef-8b96-543f20f6e61a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:54.835300 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033f8b8-881d-43ef-8b96-543f20f6e61a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:54.836560 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033f8b8-881d-43ef-8b96-543f20f6e61a service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:54.874958 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1187/2354] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:15:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:15:58.899278 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b980f-0193-4d60-afc4-e234f6e9e1dd None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:59.006673 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:59.063487 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b980f-0193-4d60-afc4-e234f6e9e1dd None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:59.072153 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0d803-7747-46e6-9a21-ef1b3b275917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:59.073277 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0d803-7747-46e6-9a21-ef1b3b275917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:59.073664 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0d803-7747-46e6-9a21-ef1b3b275917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:59.074409 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0d803-7747-46e6-9a21-ef1b3b275917 admin admin] PATH_INFO: `/v3/users/4d5875a4081c4c61a5c0208278f8edc1` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:59.076912 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0d803-7747-46e6-9a21-ef1b3b275917 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d5875a4081c4c61a5c0208278f8edc1)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:59.080265 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0d803-7747-46e6-9a21-ef1b3b275917 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:59.184807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f0e0d803-7747-46e6-9a21-ef1b3b275917 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d5875a4081c4c61a5c0208278f8edc1'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:59.211079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f0e0d803-7747-46e6-9a21-ef1b3b275917 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d5875a4081c4c61a5c0208278f8edc1'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:59.228427 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f0e0d803-7747-46e6-9a21-ef1b3b275917 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d5875a4081c4c61a5c0208278f8edc1'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:59.245993 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f0e0d803-7747-46e6-9a21-ef1b3b275917 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d5875a4081c4c61a5c0208278f8edc1'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:59.246353 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f0e0d803-7747-46e6-9a21-ef1b3b275917 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d5875a4081c4c61a5c0208278f8edc1'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:59.262748 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1168/2355] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:15:58 2022] DELETE /identity/v3/users/4d5875a4081c4c61a5c0208278f8edc1 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:59.266160 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c09d65-f01d-4346-af9e-8ecb05aba1a6 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:59.267070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c09d65-f01d-4346-af9e-8ecb05aba1a6 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:59.267466 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c09d65-f01d-4346-af9e-8ecb05aba1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:59.310178 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-61c09d65-f01d-4346-af9e-8ecb05aba1a6 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:59.312643 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-61c09d65-f01d-4346-af9e-8ecb05aba1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:59.420107 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:59.482413 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1188/2356] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:59.493163 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0d803-7747-46e6-9a21-ef1b3b275917 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:59.496143 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f0e0d803-7747-46e6-9a21-ef1b3b275917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:59.655655 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0d803-7747-46e6-9a21-ef1b3b275917 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:59.664579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa9485-538b-4b93-bc18-e6e90dac5863 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:59.665977 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa9485-538b-4b93-bc18-e6e90dac5863 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:59.666755 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa9485-538b-4b93-bc18-e6e90dac5863 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:59.667737 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa9485-538b-4b93-bc18-e6e90dac5863 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:59.690244 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aa9485-538b-4b93-bc18-e6e90dac5863 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:59.692049 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aa9485-538b-4b93-bc18-e6e90dac5863 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:59.714736 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1169/2357] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:15:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:16:00.815918 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c09d65-f01d-4346-af9e-8ecb05aba1a6 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:16:00.818834 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-61c09d65-f01d-4346-af9e-8ecb05aba1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:16:00.855648 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c09d65-f01d-4346-af9e-8ecb05aba1a6 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:16:00.876510 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d287d5b-9f7c-494b-9d10-82640c593a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:16:00.877849 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d287d5b-9f7c-494b-9d10-82640c593a54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:00.887564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d287d5b-9f7c-494b-9d10-82640c593a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:00.888472 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d287d5b-9f7c-494b-9d10-82640c593a54 admin admin] PATH_INFO: `/v3/projects/59639e2c7ad9463990169380a66e682e` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:00.894118 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d287d5b-9f7c-494b-9d10-82640c593a54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59639e2c7ad9463990169380a66e682e)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:16:00.898550 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d287d5b-9f7c-494b-9d10-82640c593a54 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:16:01.008711 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-1d287d5b-9f7c-494b-9d10-82640c593a54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59639e2c7ad9463990169380a66e682e', 'request_id': 'req-1d287d5b-9f7c-494b-9d10-82640c593a54'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:16:01.026085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-1d287d5b-9f7c-494b-9d10-82640c593a54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59639e2c7ad9463990169380a66e682e', 'request_id': 'req-1d287d5b-9f7c-494b-9d10-82640c593a54'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:16:01.046776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-1d287d5b-9f7c-494b-9d10-82640c593a54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59639e2c7ad9463990169380a66e682e', 'request_id': 'req-1d287d5b-9f7c-494b-9d10-82640c593a54'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:16:01.068841 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-1d287d5b-9f7c-494b-9d10-82640c593a54 admin admin] The token cache is being invalidate because project 59639e2c7ad9463990169380a66e682e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:16:01.069494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-1d287d5b-9f7c-494b-9d10-82640c593a54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:16:01.072326 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-1d287d5b-9f7c-494b-9d10-82640c593a54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:16:01.076273 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1189/2358] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:16:00 2022] DELETE /identity/v3/projects/59639e2c7ad9463990169380a66e682e => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:16:04.901808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a704e9e-8d9f-4085-a436-b7ce7836fb09 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:04.901808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a704e9e-8d9f-4085-a436-b7ce7836fb09 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:04.901808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a704e9e-8d9f-4085-a436-b7ce7836fb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:04.923591 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9a704e9e-8d9f-4085-a436-b7ce7836fb09 None None] MFA Rules not processed for user `2c2e0e19880f4b048faf034dd38c7365`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:16:04.925135 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9a704e9e-8d9f-4085-a436-b7ce7836fb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:16:04.997987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1170/2359] 10.222.0.17 () {58 vars in 967 bytes} [Thu Jun 16 16:16:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:16:05.020023 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d287d5b-9f7c-494b-9d10-82640c593a54 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:16:05.025640 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-1d287d5b-9f7c-494b-9d10-82640c593a54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:16:05.131865 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d287d5b-9f7c-494b-9d10-82640c593a54 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:16:05.138348 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9164c8e-c7a9-46a5-a37d-3c9b027160d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0082e37ce54f49f7837601349508c89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:16:05.139193 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9164c8e-c7a9-46a5-a37d-3c9b027160d4 service swift] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:05.139568 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9164c8e-c7a9-46a5-a37d-3c9b027160d4 service swift] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:05.140055 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9164c8e-c7a9-46a5-a37d-3c9b027160d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:05.151354 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9164c8e-c7a9-46a5-a37d-3c9b027160d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:16:05.153273 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9164c8e-c7a9-46a5-a37d-3c9b027160d4 service swift] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:16:05.167714 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1190/2360] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 16:16:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:16:07.811793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a704e9e-8d9f-4085-a436-b7ce7836fb09 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:16:07.814251 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9a704e9e-8d9f-4085-a436-b7ce7836fb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:16:07.827790 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:16:07.941202 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a704e9e-8d9f-4085-a436-b7ce7836fb09 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:16:07.951616 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e08d5-e306-4a2c-bfbb-993f6f5398b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:16:07.952862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e08d5-e306-4a2c-bfbb-993f6f5398b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:07.953269 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e08d5-e306-4a2c-bfbb-993f6f5398b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:07.953778 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e08d5-e306-4a2c-bfbb-993f6f5398b4 admin admin] PATH_INFO: `/v3/users/246ac84c69894c08b870c4579cabd042` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:07.955613 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412e08d5-e306-4a2c-bfbb-993f6f5398b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=246ac84c69894c08b870c4579cabd042)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:16:07.957765 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412e08d5-e306-4a2c-bfbb-993f6f5398b4 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:16:08.037990 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-412e08d5-e306-4a2c-bfbb-993f6f5398b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '246ac84c69894c08b870c4579cabd042'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:16:08.062602 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-412e08d5-e306-4a2c-bfbb-993f6f5398b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '246ac84c69894c08b870c4579cabd042'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:16:08.072946 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-412e08d5-e306-4a2c-bfbb-993f6f5398b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '246ac84c69894c08b870c4579cabd042'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:16:08.088141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-412e08d5-e306-4a2c-bfbb-993f6f5398b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '246ac84c69894c08b870c4579cabd042'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:16:08.089480 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-412e08d5-e306-4a2c-bfbb-993f6f5398b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '246ac84c69894c08b870c4579cabd042'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:16:08.102714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1171/2361] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:16:07 2022] DELETE /identity/v3/users/246ac84c69894c08b870c4579cabd042 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:16:08.607800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9164c8e-c7a9-46a5-a37d-3c9b027160d4 service swift] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:16:08.629996 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9164c8e-c7a9-46a5-a37d-3c9b027160d4 service swift] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:16:08.754576 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25652a8d-5671-4b7b-9872-897debe5b78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:16:08.756805 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25652a8d-5671-4b7b-9872-897debe5b78c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:08.757356 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25652a8d-5671-4b7b-9872-897debe5b78c admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:08.758169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25652a8d-5671-4b7b-9872-897debe5b78c admin admin] PATH_INFO: `/v3/projects/7b4ba60390dd4354b869bb227ef030b6` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:08.760604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25652a8d-5671-4b7b-9872-897debe5b78c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b4ba60390dd4354b869bb227ef030b6)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:16:08.763662 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25652a8d-5671-4b7b-9872-897debe5b78c admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:16:08.766368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:16:08.857888 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-25652a8d-5671-4b7b-9872-897debe5b78c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b4ba60390dd4354b869bb227ef030b6', 'request_id': 'req-25652a8d-5671-4b7b-9872-897debe5b78c'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:16:08.873173 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-25652a8d-5671-4b7b-9872-897debe5b78c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b4ba60390dd4354b869bb227ef030b6', 'request_id': 'req-25652a8d-5671-4b7b-9872-897debe5b78c'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:16:08.889663 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-25652a8d-5671-4b7b-9872-897debe5b78c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b4ba60390dd4354b869bb227ef030b6', 'request_id': 'req-25652a8d-5671-4b7b-9872-897debe5b78c'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:16:08.919379 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-25652a8d-5671-4b7b-9872-897debe5b78c admin admin] The token cache is being invalidate because project 7b4ba60390dd4354b869bb227ef030b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:16:08.919871 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-25652a8d-5671-4b7b-9872-897debe5b78c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:16:08.921223 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-25652a8d-5671-4b7b-9872-897debe5b78c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:16:08.924675 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1191/2362] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:16:08 2022] DELETE /identity/v3/projects/7b4ba60390dd4354b869bb227ef030b6 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:16:08.934303 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b52d0f-4f10-4408-b8a5-6edfbb1844c8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:08.935626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b52d0f-4f10-4408-b8a5-6edfbb1844c8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:08.936290 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b52d0f-4f10-4408-b8a5-6edfbb1844c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:08.966719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c4b52d0f-4f10-4408-b8a5-6edfbb1844c8 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:16:08.967930 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c4b52d0f-4f10-4408-b8a5-6edfbb1844c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:16:09.081622 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1172/2363] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:16:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:16:09.094707 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6266e-4521-415e-a5b6-5689ce39e1e9 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:09.095206 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6266e-4521-415e-a5b6-5689ce39e1e9 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:09.095695 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6266e-4521-415e-a5b6-5689ce39e1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:09.130264 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-41b6266e-4521-415e-a5b6-5689ce39e1e9 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:16:09.133467 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-41b6266e-4521-415e-a5b6-5689ce39e1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:16:09.147691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1192/2364] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:16:09.155187 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b52d0f-4f10-4408-b8a5-6edfbb1844c8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:16:09.172640 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b52d0f-4f10-4408-b8a5-6edfbb1844c8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:16:09.181307 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5867b1-f033-44d4-96b7-44cf359eb975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:16:09.182422 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5867b1-f033-44d4-96b7-44cf359eb975 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:09.182874 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5867b1-f033-44d4-96b7-44cf359eb975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:09.183471 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5867b1-f033-44d4-96b7-44cf359eb975 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:09.184699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5867b1-f033-44d4-96b7-44cf359eb975 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:16:09.185156 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5867b1-f033-44d4-96b7-44cf359eb975 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:16:09.188026 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5867b1-f033-44d4-96b7-44cf359eb975 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:16:09.213591 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1173/2365] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:16:09.220481 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b6266e-4521-415e-a5b6-5689ce39e1e9 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:16:09.238463 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b6266e-4521-415e-a5b6-5689ce39e1e9 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:16:09.254111 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d4144f-1201-4925-b3e4-7f2ad442b494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:16:09.257581 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d4144f-1201-4925-b3e4-7f2ad442b494 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:09.258437 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d4144f-1201-4925-b3e4-7f2ad442b494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:09.258937 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d4144f-1201-4925-b3e4-7f2ad442b494 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:09.260469 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d4144f-1201-4925-b3e4-7f2ad442b494 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:16:09.263714 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d4144f-1201-4925-b3e4-7f2ad442b494 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:16:09.315087 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1193/2366] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:16:09 2022] POST /identity/v3/projects => generated 367 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:16:09.321025 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5867b1-f033-44d4-96b7-44cf359eb975 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:16:09.336047 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5867b1-f033-44d4-96b7-44cf359eb975 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:16:09.358017 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:16:09.484790 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366ed1f-af9c-43c0-bb57-94a3e96b54fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:16:09.486258 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366ed1f-af9c-43c0-bb57-94a3e96b54fb admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:09.487147 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366ed1f-af9c-43c0-bb57-94a3e96b54fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:09.487758 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366ed1f-af9c-43c0-bb57-94a3e96b54fb admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:09.488884 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a366ed1f-af9c-43c0-bb57-94a3e96b54fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:16:09.494258 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a366ed1f-af9c-43c0-bb57-94a3e96b54fb admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:16:09.534448 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1174/2367] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:16:09 2022] POST /identity/v3/users => generated 284 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:16:09.539802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d4144f-1201-4925-b3e4-7f2ad442b494 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:16:09.561623 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d4144f-1201-4925-b3e4-7f2ad442b494 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:16:09.570374 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5dd9e-d645-4cbc-9903-8504ea7241be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:16:09.571578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5dd9e-d645-4cbc-9903-8504ea7241be admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:09.572104 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5dd9e-d645-4cbc-9903-8504ea7241be admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:09.572601 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5dd9e-d645-4cbc-9903-8504ea7241be admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:09.574070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5dd9e-d645-4cbc-9903-8504ea7241be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:16:09.577317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5dd9e-d645-4cbc-9903-8504ea7241be admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:16:09.592877 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1194/2368] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:16:09.600794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366ed1f-af9c-43c0-bb57-94a3e96b54fb admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:16:09.621548 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366ed1f-af9c-43c0-bb57-94a3e96b54fb admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:16:09.628544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed90312-48c8-4d76-9498-c4bf0e1c6836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:16:09.629947 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed90312-48c8-4d76-9498-c4bf0e1c6836 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:09.630468 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed90312-48c8-4d76-9498-c4bf0e1c6836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:09.631542 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed90312-48c8-4d76-9498-c4bf0e1c6836 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:09.633432 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed90312-48c8-4d76-9498-c4bf0e1c6836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:16:09.636565 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed90312-48c8-4d76-9498-c4bf0e1c6836 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:16:09.663972 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1175/2369] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:16:09.666410 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5dd9e-d645-4cbc-9903-8504ea7241be admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:16:09.684101 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5dd9e-d645-4cbc-9903-8504ea7241be admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:16:09.696591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbe4b5-217a-4e66-b4f3-2b3959e2dc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:16:09.698827 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbe4b5-217a-4e66-b4f3-2b3959e2dc0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:09.699349 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbe4b5-217a-4e66-b4f3-2b3959e2dc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:09.699870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbe4b5-217a-4e66-b4f3-2b3959e2dc0a admin admin] PATH_INFO: `/v3/projects/666d2b4e3bc6461e8b894c3593a6e0b3/users/4e8d02e0705349e6be5acd6923e0cc1b/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:09.722696 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dbe4b5-217a-4e66-b4f3-2b3959e2dc0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=666d2b4e3bc6461e8b894c3593a6e0b3, user_id=4e8d02e0705349e6be5acd6923e0cc1b, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:16:09.726309 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dbe4b5-217a-4e66-b4f3-2b3959e2dc0a admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:16:09.749467 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1195/2370] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:16:09 2022] PUT /identity/v3/projects/666d2b4e3bc6461e8b894c3593a6e0b3/users/4e8d02e0705349e6be5acd6923e0cc1b/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:16:09.768314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b039ee-1834-4189-8c17-9b7d112e7c87 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:09.768830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b039ee-1834-4189-8c17-9b7d112e7c87 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:09.769347 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b039ee-1834-4189-8c17-9b7d112e7c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:09.823846 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-24b039ee-1834-4189-8c17-9b7d112e7c87 None None] MFA Rules not processed for user `4e8d02e0705349e6be5acd6923e0cc1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:16:09.825570 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-24b039ee-1834-4189-8c17-9b7d112e7c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:16:09.931752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1176/2371] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:16:09 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:16:09.948554 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbe4b5-217a-4e66-b4f3-2b3959e2dc0a admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:16:09.950805 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-f1dbe4b5-217a-4e66-b4f3-2b3959e2dc0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:16:10.029598 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbe4b5-217a-4e66-b4f3-2b3959e2dc0a admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:16:10.033418 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564969c-dc20-4790-a1cc-88bbdef5952a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f11beca9c5148eea4cc45f8dc89dbda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:16:10.034201 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564969c-dc20-4790-a1cc-88bbdef5952a service glance] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:10.034368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564969c-dc20-4790-a1cc-88bbdef5952a service glance] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:10.034813 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564969c-dc20-4790-a1cc-88bbdef5952a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:10.048803 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f564969c-dc20-4790-a1cc-88bbdef5952a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:16:10.050630 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f564969c-dc20-4790-a1cc-88bbdef5952a service glance] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:16:10.066063 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1196/2372] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:16:09 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:16:15.375034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b039ee-1834-4189-8c17-9b7d112e7c87 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:16:15.377210 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-24b039ee-1834-4189-8c17-9b7d112e7c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:16:15.471315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b039ee-1834-4189-8c17-9b7d112e7c87 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:16:15.478863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297a3ec-c6e0-40e7-80dc-e8ccf2272063 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:16:15.479997 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297a3ec-c6e0-40e7-80dc-e8ccf2272063 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:15.480504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297a3ec-c6e0-40e7-80dc-e8ccf2272063 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:15.480998 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297a3ec-c6e0-40e7-80dc-e8ccf2272063 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:15.484152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e297a3ec-c6e0-40e7-80dc-e8ccf2272063 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:16:15.550751 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:16:15.596540 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297a3ec-c6e0-40e7-80dc-e8ccf2272063 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:16:15.598269 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297a3ec-c6e0-40e7-80dc-e8ccf2272063 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:16:15.610927 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1177/2373] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:16:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:16:20.057236 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=77,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:16:25.605780 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=13 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:16:39.865129 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564969c-dc20-4790-a1cc-88bbdef5952a service glance] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:16:39.870229 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-f564969c-dc20-4790-a1cc-88bbdef5952a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:16:39.955880 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564969c-dc20-4790-a1cc-88bbdef5952a service glance] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:16:39.963889 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7439a-cf29-4db1-9761-d97660b29cda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:16:39.964908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7439a-cf29-4db1-9761-d97660b29cda service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:39.965229 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7439a-cf29-4db1-9761-d97660b29cda service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:39.965542 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7439a-cf29-4db1-9761-d97660b29cda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:39.968633 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-d3c7439a-cf29-4db1-9761-d97660b29cda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:16:40.106830 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7439a-cf29-4db1-9761-d97660b29cda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:16:40.108381 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7439a-cf29-4db1-9761-d97660b29cda service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:16:40.136779 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1197/2374] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:16:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:16:45.449634 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297a3ec-c6e0-40e7-80dc-e8ccf2272063 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:16:45.452063 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e297a3ec-c6e0-40e7-80dc-e8ccf2272063 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:16:45.468958 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297a3ec-c6e0-40e7-80dc-e8ccf2272063 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:16:45.474657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3096d402-f9c2-4d3d-a0d7-0b82f9f70589 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:16:45.475609 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096d402-f9c2-4d3d-a0d7-0b82f9f70589 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:45.475787 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096d402-f9c2-4d3d-a0d7-0b82f9f70589 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:45.476152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096d402-f9c2-4d3d-a0d7-0b82f9f70589 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:45.478785 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3096d402-f9c2-4d3d-a0d7-0b82f9f70589 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:16:45.499689 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3096d402-f9c2-4d3d-a0d7-0b82f9f70589 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:16:45.501846 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3096d402-f9c2-4d3d-a0d7-0b82f9f70589 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:16:45.518896 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1178/2375] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:16:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:16:47.435788 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7439a-cf29-4db1-9761-d97660b29cda service cinder] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:16:47.545624 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7439a-cf29-4db1-9761-d97660b29cda service cinder] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:16:47.554583 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af410b16-11a7-4937-b19b-ca9a0796f6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:16:47.555695 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af410b16-11a7-4937-b19b-ca9a0796f6d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:47.556108 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af410b16-11a7-4937-b19b-ca9a0796f6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:47.556564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af410b16-11a7-4937-b19b-ca9a0796f6d8 admin admin] PATH_INFO: `/v3/users/4e8d02e0705349e6be5acd6923e0cc1b` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:47.558209 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af410b16-11a7-4937-b19b-ca9a0796f6d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e8d02e0705349e6be5acd6923e0cc1b)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:16:47.560497 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af410b16-11a7-4937-b19b-ca9a0796f6d8 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:16:47.597814 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:16:47.656494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-af410b16-11a7-4937-b19b-ca9a0796f6d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e8d02e0705349e6be5acd6923e0cc1b'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:16:47.671569 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-af410b16-11a7-4937-b19b-ca9a0796f6d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e8d02e0705349e6be5acd6923e0cc1b'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:16:47.699653 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-af410b16-11a7-4937-b19b-ca9a0796f6d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e8d02e0705349e6be5acd6923e0cc1b'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:16:47.700707 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-af410b16-11a7-4937-b19b-ca9a0796f6d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e8d02e0705349e6be5acd6923e0cc1b'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:16:47.710903 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-af410b16-11a7-4937-b19b-ca9a0796f6d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e8d02e0705349e6be5acd6923e0cc1b'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:16:47.722559 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1198/2376] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:16:47 2022] DELETE /identity/v3/users/4e8d02e0705349e6be5acd6923e0cc1b => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:16:47.730385 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a67fd-c319-4be9-b348-c6a9696bddab None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:47.731160 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a67fd-c319-4be9-b348-c6a9696bddab None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:47.731310 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a67fd-c319-4be9-b348-c6a9696bddab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:47.767799 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-a21a67fd-c319-4be9-b348-c6a9696bddab None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:16:47.770198 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a21a67fd-c319-4be9-b348-c6a9696bddab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:16:47.893294 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1179/2377] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:16:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:16:47.913539 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af410b16-11a7-4937-b19b-ca9a0796f6d8 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:16:47.916224 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-af410b16-11a7-4937-b19b-ca9a0796f6d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:16:48.097540 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af410b16-11a7-4937-b19b-ca9a0796f6d8 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:16:48.116386 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628cb86f-856c-429f-a66b-31a059ed6ba6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:16:48.117496 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cb86f-856c-429f-a66b-31a059ed6ba6 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:48.117952 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cb86f-856c-429f-a66b-31a059ed6ba6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:48.119284 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cb86f-856c-429f-a66b-31a059ed6ba6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:48.160359 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628cb86f-856c-429f-a66b-31a059ed6ba6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:16:48.163731 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628cb86f-856c-429f-a66b-31a059ed6ba6 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:16:48.187369 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1199/2378] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:16:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:16:48.888667 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a67fd-c319-4be9-b348-c6a9696bddab None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:16:48.894682 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a21a67fd-c319-4be9-b348-c6a9696bddab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:16:48.928479 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a67fd-c319-4be9-b348-c6a9696bddab None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:16:48.944757 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5613d19-d8c4-485a-849e-6b10add6ecd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:16:48.946346 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5613d19-d8c4-485a-849e-6b10add6ecd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:48.947696 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5613d19-d8c4-485a-849e-6b10add6ecd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:48.948300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5613d19-d8c4-485a-849e-6b10add6ecd8 admin admin] PATH_INFO: `/v3/projects/666d2b4e3bc6461e8b894c3593a6e0b3` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:48.953489 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5613d19-d8c4-485a-849e-6b10add6ecd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=666d2b4e3bc6461e8b894c3593a6e0b3)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:16:48.961241 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5613d19-d8c4-485a-849e-6b10add6ecd8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:16:49.056588 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e5613d19-d8c4-485a-849e-6b10add6ecd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '666d2b4e3bc6461e8b894c3593a6e0b3', 'request_id': 'req-e5613d19-d8c4-485a-849e-6b10add6ecd8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:16:49.071104 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e5613d19-d8c4-485a-849e-6b10add6ecd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '666d2b4e3bc6461e8b894c3593a6e0b3', 'request_id': 'req-e5613d19-d8c4-485a-849e-6b10add6ecd8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:16:49.088757 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e5613d19-d8c4-485a-849e-6b10add6ecd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '666d2b4e3bc6461e8b894c3593a6e0b3', 'request_id': 'req-e5613d19-d8c4-485a-849e-6b10add6ecd8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:16:49.109484 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e5613d19-d8c4-485a-849e-6b10add6ecd8 admin admin] The token cache is being invalidate because project 666d2b4e3bc6461e8b894c3593a6e0b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:16:49.109484 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e5613d19-d8c4-485a-849e-6b10add6ecd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:16:49.110036 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e5613d19-d8c4-485a-849e-6b10add6ecd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:16:49.114017 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1180/2379] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:16:48 2022] DELETE /identity/v3/projects/666d2b4e3bc6461e8b894c3593a6e0b3 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:16:49.121576 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f577e-bdb8-4e60-b3d8-ed824f4e603f None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:49.122921 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f577e-bdb8-4e60-b3d8-ed824f4e603f None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:49.123215 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f577e-bdb8-4e60-b3d8-ed824f4e603f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:49.157657 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-1b7f577e-bdb8-4e60-b3d8-ed824f4e603f None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:16:49.160122 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-1b7f577e-bdb8-4e60-b3d8-ed824f4e603f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:16:49.294419 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1200/2380] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:16:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:16:49.310569 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b7a33-924d-456a-a015-f644411e67b7 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:49.311401 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b7a33-924d-456a-a015-f644411e67b7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:49.311998 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b7a33-924d-456a-a015-f644411e67b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:49.344839 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-2c5b7a33-924d-456a-a015-f644411e67b7 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:16:49.347271 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2c5b7a33-924d-456a-a015-f644411e67b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:16:49.369976 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1181/2381] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:16:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:16:49.374653 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f577e-bdb8-4e60-b3d8-ed824f4e603f None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:16:49.397251 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f577e-bdb8-4e60-b3d8-ed824f4e603f None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:16:49.408024 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c73eb2-742b-4962-9e8c-b462a0a6312b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:16:49.410181 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c73eb2-742b-4962-9e8c-b462a0a6312b admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:49.411605 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c73eb2-742b-4962-9e8c-b462a0a6312b admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:49.412526 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c73eb2-742b-4962-9e8c-b462a0a6312b admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:49.414533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c73eb2-742b-4962-9e8c-b462a0a6312b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:16:49.415258 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c73eb2-742b-4962-9e8c-b462a0a6312b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:16:49.418527 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c73eb2-742b-4962-9e8c-b462a0a6312b admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:16:49.471381 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1201/2382] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:16:49.475069 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b7a33-924d-456a-a015-f644411e67b7 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:16:49.500063 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b7a33-924d-456a-a015-f644411e67b7 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:16:49.511424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529e684-cf61-4cfd-b76d-914a3dfa6c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:16:49.512228 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529e684-cf61-4cfd-b76d-914a3dfa6c0d admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:49.513414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529e684-cf61-4cfd-b76d-914a3dfa6c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:49.513414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529e684-cf61-4cfd-b76d-914a3dfa6c0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:49.514756 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529e684-cf61-4cfd-b76d-914a3dfa6c0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:16:49.518727 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529e684-cf61-4cfd-b76d-914a3dfa6c0d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:16:49.585664 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1182/2383] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:16:49 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:16:49.592797 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c73eb2-742b-4962-9e8c-b462a0a6312b admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:16:49.622091 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c73eb2-742b-4962-9e8c-b462a0a6312b admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:16:49.688889 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:16:49.769288 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51dfbe1-9d50-43b9-a2f1-a1a052e9379c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:16:49.771232 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51dfbe1-9d50-43b9-a2f1-a1a052e9379c admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:49.771758 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51dfbe1-9d50-43b9-a2f1-a1a052e9379c admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:49.772247 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51dfbe1-9d50-43b9-a2f1-a1a052e9379c admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:49.773850 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51dfbe1-9d50-43b9-a2f1-a1a052e9379c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:16:49.776999 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51dfbe1-9d50-43b9-a2f1-a1a052e9379c admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:16:49.820068 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1202/2384] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:16:49 2022] POST /identity/v3/users => generated 292 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:16:49.822622 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529e684-cf61-4cfd-b76d-914a3dfa6c0d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:16:49.838721 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529e684-cf61-4cfd-b76d-914a3dfa6c0d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:16:49.847765 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd719d-be4e-4aa0-b570-627003781846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:16:49.849602 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd719d-be4e-4aa0-b570-627003781846 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:49.850251 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd719d-be4e-4aa0-b570-627003781846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:49.851561 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd719d-be4e-4aa0-b570-627003781846 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:49.852889 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbd719d-be4e-4aa0-b570-627003781846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:16:49.857753 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbd719d-be4e-4aa0-b570-627003781846 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:16:49.883770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1183/2385] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:16:49.888509 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51dfbe1-9d50-43b9-a2f1-a1a052e9379c admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:16:49.904027 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51dfbe1-9d50-43b9-a2f1-a1a052e9379c admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:16:49.910333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a9fd7-4fe8-41b9-89e2-b0e043aedcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:16:49.911988 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a9fd7-4fe8-41b9-89e2-b0e043aedcac admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:49.912375 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a9fd7-4fe8-41b9-89e2-b0e043aedcac admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:49.912907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a9fd7-4fe8-41b9-89e2-b0e043aedcac admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:49.913871 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a9fd7-4fe8-41b9-89e2-b0e043aedcac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:16:49.916184 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a9fd7-4fe8-41b9-89e2-b0e043aedcac admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:16:49.930967 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1203/2386] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:16:49.938796 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd719d-be4e-4aa0-b570-627003781846 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:16:49.955494 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd719d-be4e-4aa0-b570-627003781846 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:16:49.964126 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306239f9-7d7a-4587-abd3-80240d905998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:16:49.965761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306239f9-7d7a-4587-abd3-80240d905998 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:49.966513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306239f9-7d7a-4587-abd3-80240d905998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:49.967030 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306239f9-7d7a-4587-abd3-80240d905998 admin admin] PATH_INFO: `/v3/projects/cb4bbded2b5a4d5fa7760540cfa123f7/users/ff118e281f40435d8a8253ad10a9042f/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:49.996176 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306239f9-7d7a-4587-abd3-80240d905998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb4bbded2b5a4d5fa7760540cfa123f7, user_id=ff118e281f40435d8a8253ad10a9042f, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:16:50.000106 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306239f9-7d7a-4587-abd3-80240d905998 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:16:50.026809 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1184/2387] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:16:49 2022] PUT /identity/v3/projects/cb4bbded2b5a4d5fa7760540cfa123f7/users/ff118e281f40435d8a8253ad10a9042f/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:16:50.033298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9bec2-0996-4c06-a365-5a92256dd4a5 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:50.033676 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9bec2-0996-4c06-a365-5a92256dd4a5 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:50.034104 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9bec2-0996-4c06-a365-5a92256dd4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:50.065687 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-a7e9bec2-0996-4c06-a365-5a92256dd4a5 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:16:50.067682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-a7e9bec2-0996-4c06-a365-5a92256dd4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:16:50.176041 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1204/2388] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:16:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:16:50.191824 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306239f9-7d7a-4587-abd3-80240d905998 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:16:50.196142 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-306239f9-7d7a-4587-abd3-80240d905998 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:16:50.215630 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:16:50.303752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306239f9-7d7a-4587-abd3-80240d905998 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:16:50.310031 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81adc7b1-82d5-4750-b9c4-82979bfa38b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:16:50.310915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adc7b1-82d5-4750-b9c4-82979bfa38b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:50.311299 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adc7b1-82d5-4750-b9c4-82979bfa38b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:50.311698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adc7b1-82d5-4750-b9c4-82979bfa38b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:50.332734 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81adc7b1-82d5-4750-b9c4-82979bfa38b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:16:50.335860 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81adc7b1-82d5-4750-b9c4-82979bfa38b3 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:16:50.355658 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1185/2389] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:16:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:16:51.919705 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bce39a-7bde-4860-bfbc-2d41c02a3f8c None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:51.920450 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bce39a-7bde-4860-bfbc-2d41c02a3f8c None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:51.920450 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bce39a-7bde-4860-bfbc-2d41c02a3f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:51.969897 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-88bce39a-7bde-4860-bfbc-2d41c02a3f8c None None] MFA Rules not processed for user `ff118e281f40435d8a8253ad10a9042f`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:16:51.972128 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-88bce39a-7bde-4860-bfbc-2d41c02a3f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:16:52.168380 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1205/2390] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:16:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:16:52.190841 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81adc7b1-82d5-4750-b9c4-82979bfa38b3 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:16:52.195089 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-81adc7b1-82d5-4750-b9c4-82979bfa38b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:16:52.322918 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81adc7b1-82d5-4750-b9c4-82979bfa38b3 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:16:52.328885 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9c820-7c8c-4f11-b863-d80fdc91f194 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:16:52.330059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9c820-7c8c-4f11-b863-d80fdc91f194 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:52.330574 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9c820-7c8c-4f11-b863-d80fdc91f194 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:52.330969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9c820-7c8c-4f11-b863-d80fdc91f194 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:52.353675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9c820-7c8c-4f11-b863-d80fdc91f194 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:16:52.356479 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9c820-7c8c-4f11-b863-d80fdc91f194 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:16:52.375670 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1186/2391] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:16:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:16:53.980089 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632579d9-ebf7-4cc6-a98f-3dedc7181049 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:53.980708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632579d9-ebf7-4cc6-a98f-3dedc7181049 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:53.980852 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632579d9-ebf7-4cc6-a98f-3dedc7181049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:54.008667 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-632579d9-ebf7-4cc6-a98f-3dedc7181049 None None] MFA Rules not processed for user `ff118e281f40435d8a8253ad10a9042f`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:16:54.010098 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-632579d9-ebf7-4cc6-a98f-3dedc7181049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:16:54.026731 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1206/2392] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:16:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:16:54.200565 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31cb74-27b4-4b4b-ad60-a6a3f38a5608 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:54.201091 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31cb74-27b4-4b4b-ad60-a6a3f38a5608 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:54.201254 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31cb74-27b4-4b4b-ad60-a6a3f38a5608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:54.225263 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-bc31cb74-27b4-4b4b-ad60-a6a3f38a5608 None None] MFA Rules not processed for user `2c34f05634d34da29088bcf75069958d`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:16:54.227923 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bc31cb74-27b4-4b4b-ad60-a6a3f38a5608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:16:54.320747 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1187/2393] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:16:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:17:04.001972 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=82,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:17:04.310504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=74 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:17:04.819751 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632579d9-ebf7-4cc6-a98f-3dedc7181049 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:17:04.822620 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-632579d9-ebf7-4cc6-a98f-3dedc7181049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:17:04.840493 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632579d9-ebf7-4cc6-a98f-3dedc7181049 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:17:04.845985 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96def62c-3703-4572-acb1-8c05512760c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:17:04.846583 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96def62c-3703-4572-acb1-8c05512760c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:17:04.846938 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96def62c-3703-4572-acb1-8c05512760c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:17:04.847286 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96def62c-3703-4572-acb1-8c05512760c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:17:04.850148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-96def62c-3703-4572-acb1-8c05512760c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:17:05.006583 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96def62c-3703-4572-acb1-8c05512760c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:17:05.009603 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96def62c-3703-4572-acb1-8c05512760c3 service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:17:05.026202 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31cb74-27b4-4b4b-ad60-a6a3f38a5608 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:17:05.042520 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1207/2394] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:17:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:17:05.054336 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31cb74-27b4-4b4b-ad60-a6a3f38a5608 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:17:05.062460 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a4ee4-1a9f-4c36-b782-e66a05567959 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:17:05.064057 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a4ee4-1a9f-4c36-b782-e66a05567959 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:17:05.064805 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a4ee4-1a9f-4c36-b782-e66a05567959 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:17:05.065548 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a4ee4-1a9f-4c36-b782-e66a05567959 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:17:05.091455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a4ee4-1a9f-4c36-b782-e66a05567959 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:17:05.094203 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a4ee4-1a9f-4c36-b782-e66a05567959 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:17:05.107909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1188/2395] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:17:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:17:08.042009 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96def62c-3703-4572-acb1-8c05512760c3 service cinder] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:17:08.045333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-96def62c-3703-4572-acb1-8c05512760c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:17:08.219208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96def62c-3703-4572-acb1-8c05512760c3 service cinder] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:17:08.242184 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cac0b1-1223-43e8-82d7-111101d87730 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:17:08.243781 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cac0b1-1223-43e8-82d7-111101d87730 service nova] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:17:08.244302 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cac0b1-1223-43e8-82d7-111101d87730 service nova] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:17:08.245388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cac0b1-1223-43e8-82d7-111101d87730 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:17:08.251710 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-a6cac0b1-1223-43e8-82d7-111101d87730 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:17:08.269664 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cac0b1-1223-43e8-82d7-111101d87730 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:17:08.272030 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cac0b1-1223-43e8-82d7-111101d87730 service nova] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:17:08.287228 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1208/2396] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:17:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:17:14.319040 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a4ee4-1a9f-4c36-b782-e66a05567959 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:17:14.333838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a4ee4-1a9f-4c36-b782-e66a05567959 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:17:14.340384 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e16a5-bb89-426c-b017-91b807374cef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:17:14.341229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e16a5-bb89-426c-b017-91b807374cef service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:17:14.341650 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e16a5-bb89-426c-b017-91b807374cef service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:17:14.341988 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e16a5-bb89-426c-b017-91b807374cef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:17:14.345413 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-886e16a5-bb89-426c-b017-91b807374cef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:17:14.366883 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e16a5-bb89-426c-b017-91b807374cef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:17:14.369721 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e16a5-bb89-426c-b017-91b807374cef service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:17:14.385722 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1189/2397] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:17:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:17:15.619763 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cac0b1-1223-43e8-82d7-111101d87730 service nova] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:17:15.638368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cac0b1-1223-43e8-82d7-111101d87730 service nova] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:17:15.650564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c501047-3cf2-4a91-aa51-0415f00b9834 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:17:15.651469 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c501047-3cf2-4a91-aa51-0415f00b9834 service nova] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:17:15.652073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c501047-3cf2-4a91-aa51-0415f00b9834 service nova] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:17:15.652620 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c501047-3cf2-4a91-aa51-0415f00b9834 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:17:15.656262 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-5c501047-3cf2-4a91-aa51-0415f00b9834 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:17:15.787758 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c501047-3cf2-4a91-aa51-0415f00b9834 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:17:15.792751 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c501047-3cf2-4a91-aa51-0415f00b9834 service nova] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:17:15.806623 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1209/2398] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:17:15 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:17:24.377900 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=12 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:17:25.799859 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=90 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:17:55.675542 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca2af5-60f2-4f79-bb8f-04595ed9563e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:17:55.676154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca2af5-60f2-4f79-bb8f-04595ed9563e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:17:55.676303 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca2af5-60f2-4f79-bb8f-04595ed9563e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:17:55.704234 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-05ca2af5-60f2-4f79-bb8f-04595ed9563e None None] MFA Rules not processed for user `515114f0a842463fb2d1265ee33db8eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:17:55.706472 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-05ca2af5-60f2-4f79-bb8f-04595ed9563e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:17:55.719955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1190/2399] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:17:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:18:02.607588 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c501047-3cf2-4a91-aa51-0415f00b9834 service nova] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:18:02.607588 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-5c501047-3cf2-4a91-aa51-0415f00b9834 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:18:02.628478 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c501047-3cf2-4a91-aa51-0415f00b9834 service nova] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:18:02.638342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a5e2b-f7fe-4e58-abc0-530606f83b41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:18:02.639473 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a5e2b-f7fe-4e58-abc0-530606f83b41 service nova] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:18:02.639974 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a5e2b-f7fe-4e58-abc0-530606f83b41 service nova] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:18:02.640469 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a5e2b-f7fe-4e58-abc0-530606f83b41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:18:02.643758 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-2c5a5e2b-f7fe-4e58-abc0-530606f83b41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:18:02.658201 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a5e2b-f7fe-4e58-abc0-530606f83b41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:18:02.660155 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a5e2b-f7fe-4e58-abc0-530606f83b41 service nova] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:18:02.672135 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1210/2400] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:18:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:18:05.695484 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:18:11.656514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6d385-d26d-4aa5-86a6-3d4c6b08081d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:18:11.657496 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6d385-d26d-4aa5-86a6-3d4c6b08081d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:18:11.657496 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6d385-d26d-4aa5-86a6-3d4c6b08081d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:18:11.682607 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-97d6d385-d26d-4aa5-86a6-3d4c6b08081d None None] MFA Rules not processed for user `ff118e281f40435d8a8253ad10a9042f`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:18:11.684761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-97d6d385-d26d-4aa5-86a6-3d4c6b08081d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:18:11.699541 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1191/2401] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:18:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:18:12.004112 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a5e2b-f7fe-4e58-abc0-530606f83b41 service nova] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:18:12.006052 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-2c5a5e2b-f7fe-4e58-abc0-530606f83b41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:18:12.021770 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a5e2b-f7fe-4e58-abc0-530606f83b41 service nova] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:18:12.028253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a182db1-f04b-4b25-b977-35138f4f63e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:18:12.029529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a182db1-f04b-4b25-b977-35138f4f63e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:18:12.029955 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a182db1-f04b-4b25-b977-35138f4f63e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:18:12.030463 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a182db1-f04b-4b25-b977-35138f4f63e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:18:12.033652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-3a182db1-f04b-4b25-b977-35138f4f63e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:18:12.053091 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a182db1-f04b-4b25-b977-35138f4f63e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:18:12.055858 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a182db1-f04b-4b25-b977-35138f4f63e8 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:18:12.073735 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1211/2402] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:18:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:18:12.500537 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6d385-d26d-4aa5-86a6-3d4c6b08081d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:18:12.502713 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-97d6d385-d26d-4aa5-86a6-3d4c6b08081d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:18:12.624941 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6d385-d26d-4aa5-86a6-3d4c6b08081d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:18:12.631414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e18b8f-6161-4173-9aa4-efd9f08eeb27 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36b63614297346e29ff276eab444fe5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:18:12.632207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e18b8f-6161-4173-9aa4-efd9f08eeb27 service placement] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:18:12.632555 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e18b8f-6161-4173-9aa4-efd9f08eeb27 service placement] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:18:12.632836 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e18b8f-6161-4173-9aa4-efd9f08eeb27 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:18:12.635190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-67e18b8f-6161-4173-9aa4-efd9f08eeb27 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:18:12.652447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e18b8f-6161-4173-9aa4-efd9f08eeb27 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:18:12.654121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e18b8f-6161-4173-9aa4-efd9f08eeb27 service placement] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:18:12.671942 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1192/2403] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:18:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:18:21.601676 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a182db1-f04b-4b25-b977-35138f4f63e8 service neutron] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:18:21.637742 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a182db1-f04b-4b25-b977-35138f4f63e8 service neutron] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:18:21.651766 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51552bd-7b4e-47ff-bc9e-cbda14f569bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:18:21.652929 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51552bd-7b4e-47ff-bc9e-cbda14f569bc service nova] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:18:21.653474 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51552bd-7b4e-47ff-bc9e-cbda14f569bc service nova] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:18:21.654071 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51552bd-7b4e-47ff-bc9e-cbda14f569bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:18:21.656916 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-d51552bd-7b4e-47ff-bc9e-cbda14f569bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:18:21.672331 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51552bd-7b4e-47ff-bc9e-cbda14f569bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:18:21.675262 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51552bd-7b4e-47ff-bc9e-cbda14f569bc service nova] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:18:21.691432 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1212/2404] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:18:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:18:22.520239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e18b8f-6161-4173-9aa4-efd9f08eeb27 service placement] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:18:22.545069 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e18b8f-6161-4173-9aa4-efd9f08eeb27 service placement] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:18:22.556291 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec26b76-26df-49b1-92ab-5fba5448c4b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:18:22.557675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec26b76-26df-49b1-92ab-5fba5448c4b9 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:18:22.558535 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec26b76-26df-49b1-92ab-5fba5448c4b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:18:22.558877 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec26b76-26df-49b1-92ab-5fba5448c4b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:18:22.562322 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-aec26b76-26df-49b1-92ab-5fba5448c4b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:18:22.585825 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec26b76-26df-49b1-92ab-5fba5448c4b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:18:22.588605 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec26b76-26df-49b1-92ab-5fba5448c4b9 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:18:22.609681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1193/2405] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:18:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:18:26.366247 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03be369-8f3f-4118-bb7f-8ec31cd0d713 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:18:26.366889 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03be369-8f3f-4118-bb7f-8ec31cd0d713 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:18:26.367003 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03be369-8f3f-4118-bb7f-8ec31cd0d713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:18:26.375156 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=19 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:18:26.399207 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-b03be369-8f3f-4118-bb7f-8ec31cd0d713 None None] MFA Rules not processed for user `2c34f05634d34da29088bcf75069958d`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:18:26.400565 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-b03be369-8f3f-4118-bb7f-8ec31cd0d713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:18:26.409226 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1213/2406] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:18:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:18:32.600929 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=41 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:18:36.387642 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=2 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:18:36.485622 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd319b-2650-4172-aaff-b4af135156e1 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:18:36.486368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd319b-2650-4172-aaff-b4af135156e1 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:18:36.486368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd319b-2650-4172-aaff-b4af135156e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:18:36.504513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-3ecd319b-2650-4172-aaff-b4af135156e1 None None] MFA Rules not processed for user `80386d7afc994931af05fd49fa8043d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:18:36.505769 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3ecd319b-2650-4172-aaff-b4af135156e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:18:36.514852 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1194/2407] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:18:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:18:44.337878 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55556279-c321-4bab-8bf3-7ed19cbdb0e6 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:18:44.338744 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55556279-c321-4bab-8bf3-7ed19cbdb0e6 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:18:44.338744 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55556279-c321-4bab-8bf3-7ed19cbdb0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:18:44.364869 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-55556279-c321-4bab-8bf3-7ed19cbdb0e6 None None] MFA Rules not processed for user `2c2e0e19880f4b048faf034dd38c7365`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:18:44.366335 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-55556279-c321-4bab-8bf3-7ed19cbdb0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:18:44.468115 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1214/2408] 10.222.0.17 () {58 vars in 967 bytes} [Thu Jun 16 16:18:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:18:44.545712 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd319b-2650-4172-aaff-b4af135156e1 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:18:44.547182 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3ecd319b-2650-4172-aaff-b4af135156e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:18:44.679519 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd319b-2650-4172-aaff-b4af135156e1 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:18:44.686862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0170d6-8fdc-446d-9fe9-f01a20c955b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0082e37ce54f49f7837601349508c89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:18:44.688568 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0170d6-8fdc-446d-9fe9-f01a20c955b5 service swift] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:18:44.689627 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0170d6-8fdc-446d-9fe9-f01a20c955b5 service swift] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:18:44.691071 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0170d6-8fdc-446d-9fe9-f01a20c955b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:18:44.711987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0170d6-8fdc-446d-9fe9-f01a20c955b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:18:44.714173 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0170d6-8fdc-446d-9fe9-f01a20c955b5 service swift] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:18:44.726712 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1195/2409] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 16:18:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:18:53.037497 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55556279-c321-4bab-8bf3-7ed19cbdb0e6 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:18:53.055307 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55556279-c321-4bab-8bf3-7ed19cbdb0e6 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:18:53.068999 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71dba30-4cb5-4327-9f66-57adf493ebd4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36b63614297346e29ff276eab444fe5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:18:53.069804 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71dba30-4cb5-4327-9f66-57adf493ebd4 service placement] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:18:53.070197 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71dba30-4cb5-4327-9f66-57adf493ebd4 service placement] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:18:53.070484 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71dba30-4cb5-4327-9f66-57adf493ebd4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:18:53.072646 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-a71dba30-4cb5-4327-9f66-57adf493ebd4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:18:53.098786 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71dba30-4cb5-4327-9f66-57adf493ebd4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:18:53.102401 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71dba30-4cb5-4327-9f66-57adf493ebd4 service placement] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:18:53.119988 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1215/2410] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:18:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:18:54.723221 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=35 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:18:58.131664 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0170d6-8fdc-446d-9fe9-f01a20c955b5 service swift] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:18:58.134196 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8c0170d6-8fdc-446d-9fe9-f01a20c955b5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:18:58.153778 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0170d6-8fdc-446d-9fe9-f01a20c955b5 service swift] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:18:58.160537 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ebd7d-57e4-4fc5-9d55-055155328d71 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36b63614297346e29ff276eab444fe5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:18:58.161544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ebd7d-57e4-4fc5-9d55-055155328d71 service placement] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:18:58.162101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ebd7d-57e4-4fc5-9d55-055155328d71 service placement] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:18:58.162629 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ebd7d-57e4-4fc5-9d55-055155328d71 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:18:58.165005 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5c7ebd7d-57e4-4fc5-9d55-055155328d71 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:18:58.208297 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71dba30-4cb5-4327-9f66-57adf493ebd4 service placement] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:18:58.210082 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ebd7d-57e4-4fc5-9d55-055155328d71 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:18:58.210936 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-a71dba30-4cb5-4327-9f66-57adf493ebd4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:18:58.212300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ebd7d-57e4-4fc5-9d55-055155328d71 service placement] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:18:58.224282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1196/2411] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:18:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:18:58.236470 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71dba30-4cb5-4327-9f66-57adf493ebd4 service placement] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:18:58.242369 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb716c0-c0e8-4f40-9ed6-35f109cfab68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:18:58.243381 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb716c0-c0e8-4f40-9ed6-35f109cfab68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:18:58.244290 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb716c0-c0e8-4f40-9ed6-35f109cfab68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:18:58.244602 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb716c0-c0e8-4f40-9ed6-35f109cfab68 admin admin] PATH_INFO: `/v3/users/c0da50b34820422193be2b06e0da2837` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:18:58.249157 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb716c0-c0e8-4f40-9ed6-35f109cfab68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0da50b34820422193be2b06e0da2837)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:18:58.252269 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb716c0-c0e8-4f40-9ed6-35f109cfab68 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:18:58.310413 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6cb716c0-c0e8-4f40-9ed6-35f109cfab68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0da50b34820422193be2b06e0da2837'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:58.318890 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6cb716c0-c0e8-4f40-9ed6-35f109cfab68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0da50b34820422193be2b06e0da2837'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:58.332120 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6cb716c0-c0e8-4f40-9ed6-35f109cfab68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0da50b34820422193be2b06e0da2837'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:58.333327 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6cb716c0-c0e8-4f40-9ed6-35f109cfab68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0da50b34820422193be2b06e0da2837'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:58.341721 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6cb716c0-c0e8-4f40-9ed6-35f109cfab68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0da50b34820422193be2b06e0da2837'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:58.348843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1216/2412] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:18:58 2022] DELETE /identity/v3/users/c0da50b34820422193be2b06e0da2837 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:18:58.542736 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ebd7d-57e4-4fc5-9d55-055155328d71 service placement] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:18:58.567814 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ebd7d-57e4-4fc5-9d55-055155328d71 service placement] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:18:58.644911 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e141c3cb-7f14-4a82-98d5-ca958a57fa03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:18:58.645796 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141c3cb-7f14-4a82-98d5-ca958a57fa03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:18:58.646119 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141c3cb-7f14-4a82-98d5-ca958a57fa03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:18:58.646419 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141c3cb-7f14-4a82-98d5-ca958a57fa03 admin admin] PATH_INFO: `/v3/projects/925cc45674694931bfe8fc60765045c2` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:18:58.647725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e141c3cb-7f14-4a82-98d5-ca958a57fa03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=925cc45674694931bfe8fc60765045c2)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:18:58.649624 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e141c3cb-7f14-4a82-98d5-ca958a57fa03 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:18:58.707061 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e141c3cb-7f14-4a82-98d5-ca958a57fa03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '925cc45674694931bfe8fc60765045c2', 'request_id': 'req-e141c3cb-7f14-4a82-98d5-ca958a57fa03'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:58.714240 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e141c3cb-7f14-4a82-98d5-ca958a57fa03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '925cc45674694931bfe8fc60765045c2', 'request_id': 'req-e141c3cb-7f14-4a82-98d5-ca958a57fa03'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:58.723729 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e141c3cb-7f14-4a82-98d5-ca958a57fa03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '925cc45674694931bfe8fc60765045c2', 'request_id': 'req-e141c3cb-7f14-4a82-98d5-ca958a57fa03'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:58.742285 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e141c3cb-7f14-4a82-98d5-ca958a57fa03 admin admin] The token cache is being invalidate because project 925cc45674694931bfe8fc60765045c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:18:58.742573 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e141c3cb-7f14-4a82-98d5-ca958a57fa03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:58.743251 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e141c3cb-7f14-4a82-98d5-ca958a57fa03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:58.745265 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1197/2413] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:18:58 2022] DELETE /identity/v3/projects/925cc45674694931bfe8fc60765045c2 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:18:58.749207 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb716c0-c0e8-4f40-9ed6-35f109cfab68 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:18:58.750871 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-6cb716c0-c0e8-4f40-9ed6-35f109cfab68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:18:58.766192 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb716c0-c0e8-4f40-9ed6-35f109cfab68 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:18:58.851128 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2070bc-9205-43b8-bcca-ac4710acccd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:18:58.852289 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2070bc-9205-43b8-bcca-ac4710acccd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:18:58.852801 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2070bc-9205-43b8-bcca-ac4710acccd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:18:58.853135 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2070bc-9205-43b8-bcca-ac4710acccd1 admin admin] PATH_INFO: `/v3/users/d5acd259bd7648aba86df057ea081655` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:18:58.854576 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2070bc-9205-43b8-bcca-ac4710acccd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5acd259bd7648aba86df057ea081655)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:18:58.856563 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2070bc-9205-43b8-bcca-ac4710acccd1 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:18:58.862892 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:18:58.933898 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6c2070bc-9205-43b8-bcca-ac4710acccd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5acd259bd7648aba86df057ea081655'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:58.942944 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6c2070bc-9205-43b8-bcca-ac4710acccd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5acd259bd7648aba86df057ea081655'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:58.956921 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6c2070bc-9205-43b8-bcca-ac4710acccd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5acd259bd7648aba86df057ea081655'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:58.957751 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6c2070bc-9205-43b8-bcca-ac4710acccd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5acd259bd7648aba86df057ea081655'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:58.972325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6c2070bc-9205-43b8-bcca-ac4710acccd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5acd259bd7648aba86df057ea081655'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:58.979837 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1217/2414] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:18:58 2022] DELETE /identity/v3/users/d5acd259bd7648aba86df057ea081655 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:18:59.219067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e141c3cb-7f14-4a82-98d5-ca958a57fa03 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:18:59.221622 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e141c3cb-7f14-4a82-98d5-ca958a57fa03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:18:59.257491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e141c3cb-7f14-4a82-98d5-ca958a57fa03 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:18:59.330760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9b020-93b0-4f10-8220-8df0333d9b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:18:59.331788 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9b020-93b0-4f10-8220-8df0333d9b71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:18:59.332137 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9b020-93b0-4f10-8220-8df0333d9b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:18:59.332497 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9b020-93b0-4f10-8220-8df0333d9b71 admin admin] PATH_INFO: `/v3/projects/e9d65bd6f7e241859054c2ab08fa9775` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:18:59.333823 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9b020-93b0-4f10-8220-8df0333d9b71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9d65bd6f7e241859054c2ab08fa9775)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:18:59.335913 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9b020-93b0-4f10-8220-8df0333d9b71 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:18:59.367113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:18:59.412400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-24b9b020-93b0-4f10-8220-8df0333d9b71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9d65bd6f7e241859054c2ab08fa9775', 'request_id': 'req-24b9b020-93b0-4f10-8220-8df0333d9b71'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:59.423328 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-24b9b020-93b0-4f10-8220-8df0333d9b71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9d65bd6f7e241859054c2ab08fa9775', 'request_id': 'req-24b9b020-93b0-4f10-8220-8df0333d9b71'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:59.432172 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-24b9b020-93b0-4f10-8220-8df0333d9b71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9d65bd6f7e241859054c2ab08fa9775', 'request_id': 'req-24b9b020-93b0-4f10-8220-8df0333d9b71'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:59.451928 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-24b9b020-93b0-4f10-8220-8df0333d9b71 admin admin] The token cache is being invalidate because project e9d65bd6f7e241859054c2ab08fa9775 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:18:59.452407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-24b9b020-93b0-4f10-8220-8df0333d9b71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:59.453252 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-24b9b020-93b0-4f10-8220-8df0333d9b71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:59.464224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1198/2415] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:18:59 2022] DELETE /identity/v3/projects/e9d65bd6f7e241859054c2ab08fa9775 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:19:08.975226 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=24,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:19:09.446401 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:19:20.572090 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2070bc-9205-43b8-bcca-ac4710acccd1 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:19:20.574435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-6c2070bc-9205-43b8-bcca-ac4710acccd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:19:20.591262 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2070bc-9205-43b8-bcca-ac4710acccd1 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:19:20.668659 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b8f22-f8a1-4f6c-9a58-1e765847c61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:19:20.669627 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b8f22-f8a1-4f6c-9a58-1e765847c61a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:19:20.669900 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b8f22-f8a1-4f6c-9a58-1e765847c61a admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:19:20.670228 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b8f22-f8a1-4f6c-9a58-1e765847c61a admin admin] PATH_INFO: `/v3/users/ff118e281f40435d8a8253ad10a9042f` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:19:20.671726 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927b8f22-f8a1-4f6c-9a58-1e765847c61a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff118e281f40435d8a8253ad10a9042f)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:19:20.674669 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927b8f22-f8a1-4f6c-9a58-1e765847c61a admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:19:20.676983 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9b020-93b0-4f10-8220-8df0333d9b71 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:19:20.679271 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-24b9b020-93b0-4f10-8220-8df0333d9b71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:19:20.753286 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-927b8f22-f8a1-4f6c-9a58-1e765847c61a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff118e281f40435d8a8253ad10a9042f'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:19:20.775508 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9b020-93b0-4f10-8220-8df0333d9b71 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:19:20.777300 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-927b8f22-f8a1-4f6c-9a58-1e765847c61a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff118e281f40435d8a8253ad10a9042f'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:19:20.780930 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e7eef3-ffa5-41fc-853d-f44a36c4816d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:19:20.781689 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7eef3-ffa5-41fc-853d-f44a36c4816d service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:19:20.782155 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7eef3-ffa5-41fc-853d-f44a36c4816d service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:19:20.782453 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7eef3-ffa5-41fc-853d-f44a36c4816d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:19:20.785098 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-39e7eef3-ffa5-41fc-853d-f44a36c4816d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:19:20.794000 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-927b8f22-f8a1-4f6c-9a58-1e765847c61a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff118e281f40435d8a8253ad10a9042f'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:19:20.794745 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-927b8f22-f8a1-4f6c-9a58-1e765847c61a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff118e281f40435d8a8253ad10a9042f'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:19:20.801573 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e7eef3-ffa5-41fc-853d-f44a36c4816d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:19:20.803151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e7eef3-ffa5-41fc-853d-f44a36c4816d service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:19:20.809259 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-927b8f22-f8a1-4f6c-9a58-1e765847c61a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff118e281f40435d8a8253ad10a9042f'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:19:20.815282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1199/2416] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:19:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:19:20.819526 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1218/2417] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:19:20 2022] DELETE /identity/v3/users/ff118e281f40435d8a8253ad10a9042f => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:19:21.045121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e7eef3-ffa5-41fc-853d-f44a36c4816d service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:19:21.046625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-39e7eef3-ffa5-41fc-853d-f44a36c4816d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:19:21.060559 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e7eef3-ffa5-41fc-853d-f44a36c4816d service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:19:21.162273 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0d36a-d69e-4983-a0df-2c4a13ace9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:19:21.163448 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0d36a-d69e-4983-a0df-2c4a13ace9bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:19:21.163825 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0d36a-d69e-4983-a0df-2c4a13ace9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:19:21.164195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0d36a-d69e-4983-a0df-2c4a13ace9bf admin admin] PATH_INFO: `/v3/projects/cb4bbded2b5a4d5fa7760540cfa123f7` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:19:21.165707 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0d36a-d69e-4983-a0df-2c4a13ace9bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb4bbded2b5a4d5fa7760540cfa123f7)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:19:21.167995 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0d36a-d69e-4983-a0df-2c4a13ace9bf admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:19:21.240582 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-57d0d36a-d69e-4983-a0df-2c4a13ace9bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb4bbded2b5a4d5fa7760540cfa123f7', 'request_id': 'req-57d0d36a-d69e-4983-a0df-2c4a13ace9bf'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:19:21.246725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-57d0d36a-d69e-4983-a0df-2c4a13ace9bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb4bbded2b5a4d5fa7760540cfa123f7', 'request_id': 'req-57d0d36a-d69e-4983-a0df-2c4a13ace9bf'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:19:21.256991 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-57d0d36a-d69e-4983-a0df-2c4a13ace9bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb4bbded2b5a4d5fa7760540cfa123f7', 'request_id': 'req-57d0d36a-d69e-4983-a0df-2c4a13ace9bf'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:19:21.271995 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-57d0d36a-d69e-4983-a0df-2c4a13ace9bf admin admin] The token cache is being invalidate because project cb4bbded2b5a4d5fa7760540cfa123f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:19:21.272275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-57d0d36a-d69e-4983-a0df-2c4a13ace9bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:19:21.273072 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-57d0d36a-d69e-4983-a0df-2c4a13ace9bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:19:21.274585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1200/2418] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:19:21 2022] DELETE /identity/v3/projects/cb4bbded2b5a4d5fa7760540cfa123f7 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:19:24.290783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b8f22-f8a1-4f6c-9a58-1e765847c61a admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:19:24.293399 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-927b8f22-f8a1-4f6c-9a58-1e765847c61a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:19:24.320210 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b8f22-f8a1-4f6c-9a58-1e765847c61a admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:19:24.413022 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42cfb0c-c86e-41d8-ab87-11a901b939a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:19:24.413887 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cfb0c-c86e-41d8-ab87-11a901b939a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:19:24.414224 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cfb0c-c86e-41d8-ab87-11a901b939a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:19:24.414688 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cfb0c-c86e-41d8-ab87-11a901b939a1 admin admin] PATH_INFO: `/v3/users/515114f0a842463fb2d1265ee33db8eb` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:19:24.416296 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42cfb0c-c86e-41d8-ab87-11a901b939a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=515114f0a842463fb2d1265ee33db8eb)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:19:24.418349 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42cfb0c-c86e-41d8-ab87-11a901b939a1 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:19:24.451215 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:19:24.502999 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-d42cfb0c-c86e-41d8-ab87-11a901b939a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '515114f0a842463fb2d1265ee33db8eb'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:19:24.515657 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-d42cfb0c-c86e-41d8-ab87-11a901b939a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '515114f0a842463fb2d1265ee33db8eb'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:19:24.540185 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-d42cfb0c-c86e-41d8-ab87-11a901b939a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '515114f0a842463fb2d1265ee33db8eb'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:19:24.541200 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-d42cfb0c-c86e-41d8-ab87-11a901b939a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '515114f0a842463fb2d1265ee33db8eb'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:19:24.554023 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-d42cfb0c-c86e-41d8-ab87-11a901b939a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '515114f0a842463fb2d1265ee33db8eb'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:19:24.564443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1219/2419] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:19:24 2022] DELETE /identity/v3/users/515114f0a842463fb2d1265ee33db8eb => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:19:24.772584 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0d36a-d69e-4983-a0df-2c4a13ace9bf admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:19:24.774181 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-57d0d36a-d69e-4983-a0df-2c4a13ace9bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:19:24.803330 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0d36a-d69e-4983-a0df-2c4a13ace9bf admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:19:24.866899 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:19:24.906251 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2371a-1ebc-473f-8d10-1f73f2935186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:19:24.908088 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2371a-1ebc-473f-8d10-1f73f2935186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:19:24.908386 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2371a-1ebc-473f-8d10-1f73f2935186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:19:24.908671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2371a-1ebc-473f-8d10-1f73f2935186 admin admin] PATH_INFO: `/v3/projects/6eee49063a5847dba6ffb520d0149d88` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:19:24.910066 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2371a-1ebc-473f-8d10-1f73f2935186 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6eee49063a5847dba6ffb520d0149d88)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:19:24.912152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2371a-1ebc-473f-8d10-1f73f2935186 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:19:24.972011 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2aa2371a-1ebc-473f-8d10-1f73f2935186 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eee49063a5847dba6ffb520d0149d88', 'request_id': 'req-2aa2371a-1ebc-473f-8d10-1f73f2935186'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:19:24.980516 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2aa2371a-1ebc-473f-8d10-1f73f2935186 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eee49063a5847dba6ffb520d0149d88', 'request_id': 'req-2aa2371a-1ebc-473f-8d10-1f73f2935186'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:19:24.996271 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2aa2371a-1ebc-473f-8d10-1f73f2935186 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eee49063a5847dba6ffb520d0149d88', 'request_id': 'req-2aa2371a-1ebc-473f-8d10-1f73f2935186'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:19:25.010955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2aa2371a-1ebc-473f-8d10-1f73f2935186 admin admin] The token cache is being invalidate because project 6eee49063a5847dba6ffb520d0149d88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:19:25.011320 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2aa2371a-1ebc-473f-8d10-1f73f2935186 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:19:25.012101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2aa2371a-1ebc-473f-8d10-1f73f2935186 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:19:25.013876 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1201/2420] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:19:24 2022] DELETE /identity/v3/projects/6eee49063a5847dba6ffb520d0149d88 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:19:25.022437 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498267e-dbcf-484b-88bd-3e047502eb5c None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:19:25.022997 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498267e-dbcf-484b-88bd-3e047502eb5c None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:19:25.022997 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498267e-dbcf-484b-88bd-3e047502eb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:19:25.048851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-a498267e-dbcf-484b-88bd-3e047502eb5c None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:19:25.050763 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-a498267e-dbcf-484b-88bd-3e047502eb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:19:25.147239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1220/2421] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:19:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:19:25.155235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ec3c6-4486-43eb-a8cb-2e77be0cb0ae None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:19:25.155607 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ec3c6-4486-43eb-a8cb-2e77be0cb0ae None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:19:25.155906 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ec3c6-4486-43eb-a8cb-2e77be0cb0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:19:25.191103 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ef0ec3c6-4486-43eb-a8cb-2e77be0cb0ae None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:19:25.193165 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ef0ec3c6-4486-43eb-a8cb-2e77be0cb0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:19:25.203880 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1202/2422] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:19:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:19:25.208767 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a498267e-dbcf-484b-88bd-3e047502eb5c None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:19:25.219877 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a498267e-dbcf-484b-88bd-3e047502eb5c None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:19:25.225817 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7afcbc-2175-44e8-acfe-fcae9123a2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:19:25.226757 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7afcbc-2175-44e8-acfe-fcae9123a2ae admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:19:25.226979 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7afcbc-2175-44e8-acfe-fcae9123a2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:19:25.227138 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7afcbc-2175-44e8-acfe-fcae9123a2ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:19:25.227863 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7afcbc-2175-44e8-acfe-fcae9123a2ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:19:25.228177 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7afcbc-2175-44e8-acfe-fcae9123a2ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:19:25.229890 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7afcbc-2175-44e8-acfe-fcae9123a2ae admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:19:25.245821 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1221/2423] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:19:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:19:25.251834 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ec3c6-4486-43eb-a8cb-2e77be0cb0ae None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:19:25.269874 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ec3c6-4486-43eb-a8cb-2e77be0cb0ae None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:19:25.276374 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111df06d-49e6-44b3-89fb-4be23d6de9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:19:25.277235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111df06d-49e6-44b3-89fb-4be23d6de9e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:19:25.277567 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111df06d-49e6-44b3-89fb-4be23d6de9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:19:25.277860 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111df06d-49e6-44b3-89fb-4be23d6de9e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:19:25.278794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111df06d-49e6-44b3-89fb-4be23d6de9e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:19:25.282174 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111df06d-49e6-44b3-89fb-4be23d6de9e6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:19:25.329538 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1203/2424] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:19:25 2022] POST /identity/v3/projects => generated 389 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:19:25.334369 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7afcbc-2175-44e8-acfe-fcae9123a2ae admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:19:25.346535 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7afcbc-2175-44e8-acfe-fcae9123a2ae admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:19:25.413041 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382beb6d-1e63-4b83-931f-192e6f97154c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:19:25.413765 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382beb6d-1e63-4b83-931f-192e6f97154c admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:19:25.414094 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382beb6d-1e63-4b83-931f-192e6f97154c admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:19:25.414379 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382beb6d-1e63-4b83-931f-192e6f97154c admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:19:25.415126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382beb6d-1e63-4b83-931f-192e6f97154c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:19:25.417203 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382beb6d-1e63-4b83-931f-192e6f97154c admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:19:25.441639 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1222/2425] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:19:25 2022] POST /identity/v3/users => generated 295 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:19:25.446020 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111df06d-49e6-44b3-89fb-4be23d6de9e6 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:19:25.457131 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111df06d-49e6-44b3-89fb-4be23d6de9e6 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:19:25.462834 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a857530-3c59-4a87-93d5-67daa2e29ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:19:25.463520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a857530-3c59-4a87-93d5-67daa2e29ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:19:25.463829 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a857530-3c59-4a87-93d5-67daa2e29ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:19:25.464108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a857530-3c59-4a87-93d5-67daa2e29ef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:19:25.464894 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a857530-3c59-4a87-93d5-67daa2e29ef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:19:25.466645 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a857530-3c59-4a87-93d5-67daa2e29ef8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:19:25.480207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1204/2426] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:19:25.484958 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382beb6d-1e63-4b83-931f-192e6f97154c admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:19:25.497999 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382beb6d-1e63-4b83-931f-192e6f97154c admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:19:25.505687 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e3018-ecd4-43a7-9b3e-d74f59626c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:19:25.506881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e3018-ecd4-43a7-9b3e-d74f59626c55 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:19:25.507201 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e3018-ecd4-43a7-9b3e-d74f59626c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:19:25.507921 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e3018-ecd4-43a7-9b3e-d74f59626c55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:19:25.508969 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e3018-ecd4-43a7-9b3e-d74f59626c55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:19:25.510813 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e3018-ecd4-43a7-9b3e-d74f59626c55 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:19:25.528503 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1223/2427] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:19:25.533795 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a857530-3c59-4a87-93d5-67daa2e29ef8 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:19:25.546027 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a857530-3c59-4a87-93d5-67daa2e29ef8 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:19:25.552059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2b648-344b-4492-804e-14e0429aca62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:19:25.552996 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2b648-344b-4492-804e-14e0429aca62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:19:25.553283 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2b648-344b-4492-804e-14e0429aca62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:19:25.553560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2b648-344b-4492-804e-14e0429aca62 admin admin] PATH_INFO: `/v3/projects/5411e6d1c7a94aeeaaa8a0a30fb4887b/users/2b32b74cc1e44ce5a7e687e3c04e8099/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:19:25.570935 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe2b648-344b-4492-804e-14e0429aca62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5411e6d1c7a94aeeaaa8a0a30fb4887b, user_id=2b32b74cc1e44ce5a7e687e3c04e8099, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:19:25.573695 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe2b648-344b-4492-804e-14e0429aca62 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:19:25.590654 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1205/2428] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:19:25 2022] PUT /identity/v3/projects/5411e6d1c7a94aeeaaa8a0a30fb4887b/users/2b32b74cc1e44ce5a7e687e3c04e8099/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:19:25.597060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e26c7b-809c-42a5-b546-741d73f687ce None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:19:25.597370 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e26c7b-809c-42a5-b546-741d73f687ce None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:19:25.597679 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e26c7b-809c-42a5-b546-741d73f687ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:19:25.605543 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:19:25.621436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-c9e26c7b-809c-42a5-b546-741d73f687ce None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:19:25.623173 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-c9e26c7b-809c-42a5-b546-741d73f687ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:19:25.712220 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1224/2429] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:19:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:19:25.723408 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2b648-344b-4492-804e-14e0429aca62 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:19:25.725127 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-afe2b648-344b-4492-804e-14e0429aca62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:19:25.797415 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2b648-344b-4492-804e-14e0429aca62 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:19:25.801630 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eeae8b-e6db-4c4f-add6-168c962b7403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:19:25.802485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eeae8b-e6db-4c4f-add6-168c962b7403 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:19:25.802773 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eeae8b-e6db-4c4f-add6-168c962b7403 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:19:25.803096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eeae8b-e6db-4c4f-add6-168c962b7403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:19:25.810345 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:19:25.816479 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eeae8b-e6db-4c4f-add6-168c962b7403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:19:25.818422 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eeae8b-e6db-4c4f-add6-168c962b7403 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:19:25.833332 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1206/2430] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:19:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:19:30.824666 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6e881-8e68-446d-828b-2b3bdfecbea9 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:19:30.825213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6e881-8e68-446d-828b-2b3bdfecbea9 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:19:30.825316 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6e881-8e68-446d-828b-2b3bdfecbea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:19:30.867018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-9de6e881-8e68-446d-828b-2b3bdfecbea9 None None] MFA Rules not processed for user `2b32b74cc1e44ce5a7e687e3c04e8099`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:19:30.868758 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-9de6e881-8e68-446d-828b-2b3bdfecbea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:19:30.992847 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1225/2431] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:19:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:19:31.012219 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eeae8b-e6db-4c4f-add6-168c962b7403 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:19:31.013799 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a5eeae8b-e6db-4c4f-add6-168c962b7403 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:19:31.138813 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eeae8b-e6db-4c4f-add6-168c962b7403 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:19:31.149498 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac023a0-e500-4ea7-810d-22ccfe36e468 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:19:31.150345 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac023a0-e500-4ea7-810d-22ccfe36e468 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:19:31.150728 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac023a0-e500-4ea7-810d-22ccfe36e468 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:19:31.151160 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac023a0-e500-4ea7-810d-22ccfe36e468 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:19:31.162450 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac023a0-e500-4ea7-810d-22ccfe36e468 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:19:31.164158 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac023a0-e500-4ea7-810d-22ccfe36e468 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:19:31.177631 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1207/2432] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:19:31 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:19:32.725991 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293de5ed-5fae-42de-a35b-902526d402e8 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:19:32.726931 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293de5ed-5fae-42de-a35b-902526d402e8 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:19:32.726931 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293de5ed-5fae-42de-a35b-902526d402e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:19:32.761531 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-293de5ed-5fae-42de-a35b-902526d402e8 None None] MFA Rules not processed for user `2b32b74cc1e44ce5a7e687e3c04e8099`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:19:32.763623 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-293de5ed-5fae-42de-a35b-902526d402e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:19:32.775338 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1226/2433] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:19:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:19:41.171391 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=37 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:19:42.752075 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=60 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:19:56.259742 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac023a0-e500-4ea7-810d-22ccfe36e468 service nova] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:19:56.263201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8ac023a0-e500-4ea7-810d-22ccfe36e468 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:19:56.363979 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac023a0-e500-4ea7-810d-22ccfe36e468 service nova] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:19:56.369423 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100a101-986f-4f2e-8d08-26e861db9682 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36b63614297346e29ff276eab444fe5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:19:56.370320 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100a101-986f-4f2e-8d08-26e861db9682 service placement] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:19:56.370675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100a101-986f-4f2e-8d08-26e861db9682 service placement] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:19:56.371039 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100a101-986f-4f2e-8d08-26e861db9682 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:19:56.372955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b100a101-986f-4f2e-8d08-26e861db9682 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:19:56.386591 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100a101-986f-4f2e-8d08-26e861db9682 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:19:56.388337 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100a101-986f-4f2e-8d08-26e861db9682 service placement] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:19:56.400008 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1208/2434] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:19:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:20:02.533389 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293de5ed-5fae-42de-a35b-902526d402e8 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:02.543423 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293de5ed-5fae-42de-a35b-902526d402e8 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:02.547718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c634e-c4b9-48e5-a08d-8ef550a14990 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:02.548375 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c634e-c4b9-48e5-a08d-8ef550a14990 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:02.548738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c634e-c4b9-48e5-a08d-8ef550a14990 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:02.549006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c634e-c4b9-48e5-a08d-8ef550a14990 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:02.550972 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-965c634e-c4b9-48e5-a08d-8ef550a14990 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:02.564810 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c634e-c4b9-48e5-a08d-8ef550a14990 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:02.566344 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c634e-c4b9-48e5-a08d-8ef550a14990 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:02.577859 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1227/2435] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:20:06.315894 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100a101-986f-4f2e-8d08-26e861db9682 service placement] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:06.318442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b100a101-986f-4f2e-8d08-26e861db9682 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:06.336136 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100a101-986f-4f2e-8d08-26e861db9682 service placement] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:06.341074 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be9032-2a7b-4494-a108-8c9ea21cd8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:06.341991 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be9032-2a7b-4494-a108-8c9ea21cd8bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:06.342282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be9032-2a7b-4494-a108-8c9ea21cd8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:06.342585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be9032-2a7b-4494-a108-8c9ea21cd8bb admin admin] PATH_INFO: `/v3/users/2c34f05634d34da29088bcf75069958d` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:06.344671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be9032-2a7b-4494-a108-8c9ea21cd8bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c34f05634d34da29088bcf75069958d)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:06.346674 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be9032-2a7b-4494-a108-8c9ea21cd8bb admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:06.403998 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b8be9032-2a7b-4494-a108-8c9ea21cd8bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c34f05634d34da29088bcf75069958d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:06.418094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b8be9032-2a7b-4494-a108-8c9ea21cd8bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c34f05634d34da29088bcf75069958d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:06.428250 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b8be9032-2a7b-4494-a108-8c9ea21cd8bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c34f05634d34da29088bcf75069958d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:06.436175 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b8be9032-2a7b-4494-a108-8c9ea21cd8bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c34f05634d34da29088bcf75069958d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:06.436946 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b8be9032-2a7b-4494-a108-8c9ea21cd8bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c34f05634d34da29088bcf75069958d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:06.443201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1209/2436] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:20:06 2022] DELETE /identity/v3/users/2c34f05634d34da29088bcf75069958d => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:06.655716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c634e-c4b9-48e5-a08d-8ef550a14990 service neutron] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:06.666402 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c634e-c4b9-48e5-a08d-8ef550a14990 service neutron] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:06.730940 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bcf2cc-72aa-4c17-bb4a-2a7dca6b13a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:06.731818 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bcf2cc-72aa-4c17-bb4a-2a7dca6b13a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:06.732092 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bcf2cc-72aa-4c17-bb4a-2a7dca6b13a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:06.732369 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bcf2cc-72aa-4c17-bb4a-2a7dca6b13a9 admin admin] PATH_INFO: `/v3/projects/b091584c67814e4d8f7e88f0596880e3` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:06.733632 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bcf2cc-72aa-4c17-bb4a-2a7dca6b13a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b091584c67814e4d8f7e88f0596880e3)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:06.735647 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bcf2cc-72aa-4c17-bb4a-2a7dca6b13a9 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:06.791686 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-d0bcf2cc-72aa-4c17-bb4a-2a7dca6b13a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b091584c67814e4d8f7e88f0596880e3', 'request_id': 'req-d0bcf2cc-72aa-4c17-bb4a-2a7dca6b13a9'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:06.798272 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-d0bcf2cc-72aa-4c17-bb4a-2a7dca6b13a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b091584c67814e4d8f7e88f0596880e3', 'request_id': 'req-d0bcf2cc-72aa-4c17-bb4a-2a7dca6b13a9'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:06.816785 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-d0bcf2cc-72aa-4c17-bb4a-2a7dca6b13a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b091584c67814e4d8f7e88f0596880e3', 'request_id': 'req-d0bcf2cc-72aa-4c17-bb4a-2a7dca6b13a9'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:06.830534 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-d0bcf2cc-72aa-4c17-bb4a-2a7dca6b13a9 admin admin] The token cache is being invalidate because project b091584c67814e4d8f7e88f0596880e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:20:06.830813 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-d0bcf2cc-72aa-4c17-bb4a-2a7dca6b13a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:06.831499 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-d0bcf2cc-72aa-4c17-bb4a-2a7dca6b13a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:06.833153 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1228/2437] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:20:06 2022] DELETE /identity/v3/projects/b091584c67814e4d8f7e88f0596880e3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:06.841067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478dbe5-73ba-4132-b7d1-f32d9ee905c9 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:06.841359 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478dbe5-73ba-4132-b7d1-f32d9ee905c9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:06.841642 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478dbe5-73ba-4132-b7d1-f32d9ee905c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:06.870438 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-3478dbe5-73ba-4132-b7d1-f32d9ee905c9 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:06.872160 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3478dbe5-73ba-4132-b7d1-f32d9ee905c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:06.967057 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1210/2438] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:20:06.976771 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa36bdb-d060-49ed-ba3f-49c21954d40e None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:06.977263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa36bdb-d060-49ed-ba3f-49c21954d40e None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:06.977551 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa36bdb-d060-49ed-ba3f-49c21954d40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:07.001067 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-3fa36bdb-d060-49ed-ba3f-49c21954d40e None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:07.002043 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-3fa36bdb-d060-49ed-ba3f-49c21954d40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:07.013832 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1229/2439] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:07.019364 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3478dbe5-73ba-4132-b7d1-f32d9ee905c9 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:07.024334 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:07.031608 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3478dbe5-73ba-4132-b7d1-f32d9ee905c9 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:07.037501 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e46e62d-d66a-41c8-8013-615df151d5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:07.038499 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46e62d-d66a-41c8-8013-615df151d5c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:07.038984 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46e62d-d66a-41c8-8013-615df151d5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:07.039840 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46e62d-d66a-41c8-8013-615df151d5c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:07.041005 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e46e62d-d66a-41c8-8013-615df151d5c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:20:07.041138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e46e62d-d66a-41c8-8013-615df151d5c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:07.042983 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e46e62d-d66a-41c8-8013-615df151d5c0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:07.057207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1211/2440] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:20:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:20:07.064187 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa36bdb-d060-49ed-ba3f-49c21954d40e None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:07.079551 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa36bdb-d060-49ed-ba3f-49c21954d40e None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:07.084828 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bfab0-1d2d-4d1f-b82e-eaa51922f20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:07.085502 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bfab0-1d2d-4d1f-b82e-eaa51922f20c admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:07.085787 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bfab0-1d2d-4d1f-b82e-eaa51922f20c admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:07.086108 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bfab0-1d2d-4d1f-b82e-eaa51922f20c admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:07.086844 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315bfab0-1d2d-4d1f-b82e-eaa51922f20c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:07.088792 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315bfab0-1d2d-4d1f-b82e-eaa51922f20c admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:07.120957 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1230/2441] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:20:07 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:20:07.127809 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e46e62d-d66a-41c8-8013-615df151d5c0 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:07.139792 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e46e62d-d66a-41c8-8013-615df151d5c0 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:07.211167 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4b914-5f7e-4ef8-8e79-8cfc6bc87af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:07.211777 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4b914-5f7e-4ef8-8e79-8cfc6bc87af4 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:07.212034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4b914-5f7e-4ef8-8e79-8cfc6bc87af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:07.212310 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4b914-5f7e-4ef8-8e79-8cfc6bc87af4 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:07.213024 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c4b914-5f7e-4ef8-8e79-8cfc6bc87af4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:07.215286 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c4b914-5f7e-4ef8-8e79-8cfc6bc87af4 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:07.240039 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1212/2442] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:20:07 2022] POST /identity/v3/users => generated 291 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:07.246373 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bfab0-1d2d-4d1f-b82e-eaa51922f20c admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:07.265827 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bfab0-1d2d-4d1f-b82e-eaa51922f20c admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:07.272277 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb01d0-870e-43db-ba96-4f646ba1efc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:07.273028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb01d0-870e-43db-ba96-4f646ba1efc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:07.273358 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb01d0-870e-43db-ba96-4f646ba1efc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:07.273680 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb01d0-870e-43db-ba96-4f646ba1efc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:07.274569 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb01d0-870e-43db-ba96-4f646ba1efc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:07.276476 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb01d0-870e-43db-ba96-4f646ba1efc9 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:07.290295 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1231/2443] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:07.298301 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4b914-5f7e-4ef8-8e79-8cfc6bc87af4 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:07.308683 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4b914-5f7e-4ef8-8e79-8cfc6bc87af4 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:07.315227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e61eb-33d1-4181-9a89-a548639c4bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:07.316019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e61eb-33d1-4181-9a89-a548639c4bfd admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:07.316693 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e61eb-33d1-4181-9a89-a548639c4bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:07.316693 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e61eb-33d1-4181-9a89-a548639c4bfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:07.317344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e61eb-33d1-4181-9a89-a548639c4bfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:07.319379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e61eb-33d1-4181-9a89-a548639c4bfd admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:07.336632 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1213/2444] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:07.341747 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb01d0-870e-43db-ba96-4f646ba1efc9 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:07.354223 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb01d0-870e-43db-ba96-4f646ba1efc9 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:07.362142 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113d886-44be-497a-811f-220c4d775083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:07.363251 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113d886-44be-497a-811f-220c4d775083 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:07.363533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113d886-44be-497a-811f-220c4d775083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:07.363810 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113d886-44be-497a-811f-220c4d775083 admin admin] PATH_INFO: `/v3/projects/b6b6f0e060c1404798ddf9def43ee055/users/af15226f06334295bc056bf7490d9314/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:07.383209 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113d886-44be-497a-811f-220c4d775083 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6b6f0e060c1404798ddf9def43ee055, user_id=af15226f06334295bc056bf7490d9314, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:07.386351 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113d886-44be-497a-811f-220c4d775083 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:07.403936 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1232/2445] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:07 2022] PUT /identity/v3/projects/b6b6f0e060c1404798ddf9def43ee055/users/af15226f06334295bc056bf7490d9314/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:07.415722 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce6e336-043c-4016-9d2b-1eb12e81ed1d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:07.416887 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce6e336-043c-4016-9d2b-1eb12e81ed1d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:07.416887 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce6e336-043c-4016-9d2b-1eb12e81ed1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:07.456902 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-6ce6e336-043c-4016-9d2b-1eb12e81ed1d None None] MFA Rules not processed for user `af15226f06334295bc056bf7490d9314`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:07.458648 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6ce6e336-043c-4016-9d2b-1eb12e81ed1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:07.548510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1214/2446] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:07.553736 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113d886-44be-497a-811f-220c4d775083 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:07.571165 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:07.632192 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113d886-44be-497a-811f-220c4d775083 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:07.637593 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d6040-2d6b-473a-a02c-1c83c721f17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:07.638418 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d6040-2d6b-473a-a02c-1c83c721f17a admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:07.638613 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d6040-2d6b-473a-a02c-1c83c721f17a admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:07.638869 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d6040-2d6b-473a-a02c-1c83c721f17a admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:07.639577 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d6040-2d6b-473a-a02c-1c83c721f17a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:07.642066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d6040-2d6b-473a-a02c-1c83c721f17a admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:07.673370 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1233/2447] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:20:07 2022] POST /identity/v3/projects => generated 379 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:07.678170 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce6e336-043c-4016-9d2b-1eb12e81ed1d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:07.690481 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce6e336-043c-4016-9d2b-1eb12e81ed1d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:07.755638 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:07.763425 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06b767-45e8-4117-8d12-bbf56418b601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:07.764203 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06b767-45e8-4117-8d12-bbf56418b601 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:07.764502 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06b767-45e8-4117-8d12-bbf56418b601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:07.765286 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06b767-45e8-4117-8d12-bbf56418b601 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:07.766319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c06b767-45e8-4117-8d12-bbf56418b601 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:07.768291 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c06b767-45e8-4117-8d12-bbf56418b601 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:07.792317 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1215/2448] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:20:07 2022] POST /identity/v3/users => generated 296 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:07.797013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d6040-2d6b-473a-a02c-1c83c721f17a admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:07.810434 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d6040-2d6b-473a-a02c-1c83c721f17a admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:07.817411 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01161f-344b-40ae-acba-a326620acb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:07.818143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01161f-344b-40ae-acba-a326620acb28 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:07.818432 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01161f-344b-40ae-acba-a326620acb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:07.818709 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01161f-344b-40ae-acba-a326620acb28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:07.819470 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01161f-344b-40ae-acba-a326620acb28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:07.821170 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01161f-344b-40ae-acba-a326620acb28 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:07.839144 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1234/2449] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:07.840954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06b767-45e8-4117-8d12-bbf56418b601 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:07.853057 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06b767-45e8-4117-8d12-bbf56418b601 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:07.859465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14fd5e-1312-4012-bbc4-1a56c28d2fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:07.860429 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14fd5e-1312-4012-bbc4-1a56c28d2fd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:07.860719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14fd5e-1312-4012-bbc4-1a56c28d2fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:07.860992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14fd5e-1312-4012-bbc4-1a56c28d2fd0 admin admin] PATH_INFO: `/v3/projects/5e4aa3d2826a4a628007e94fcef2295b/users/13bcfaa22f2747e2be81e1238e164800/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:07.879610 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc14fd5e-1312-4012-bbc4-1a56c28d2fd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e4aa3d2826a4a628007e94fcef2295b, user_id=13bcfaa22f2747e2be81e1238e164800, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:07.882485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc14fd5e-1312-4012-bbc4-1a56c28d2fd0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:07.900416 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1216/2450] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:07 2022] PUT /identity/v3/projects/5e4aa3d2826a4a628007e94fcef2295b/users/13bcfaa22f2747e2be81e1238e164800/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:20:07.904914 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01161f-344b-40ae-acba-a326620acb28 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:07.925304 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01161f-344b-40ae-acba-a326620acb28 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:08.003478 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b013516f-fbff-46cd-8c0d-d5324beb592c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:08.005309 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013516f-fbff-46cd-8c0d-d5324beb592c admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:08.005309 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013516f-fbff-46cd-8c0d-d5324beb592c admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:08.005309 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013516f-fbff-46cd-8c0d-d5324beb592c admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:08.005815 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b013516f-fbff-46cd-8c0d-d5324beb592c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:08.007102 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b013516f-fbff-46cd-8c0d-d5324beb592c admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:08.019333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1235/2451] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:08.023733 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14fd5e-1312-4012-bbc4-1a56c28d2fd0 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:08.036437 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14fd5e-1312-4012-bbc4-1a56c28d2fd0 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:08.044582 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db71494-cb8f-45ed-932d-60cf996536d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:08.045506 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db71494-cb8f-45ed-932d-60cf996536d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:08.045809 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db71494-cb8f-45ed-932d-60cf996536d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:08.046817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db71494-cb8f-45ed-932d-60cf996536d6 admin admin] PATH_INFO: `/v3/projects/5e4aa3d2826a4a628007e94fcef2295b/users/13bcfaa22f2747e2be81e1238e164800/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:08.048969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db71494-cb8f-45ed-932d-60cf996536d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e4aa3d2826a4a628007e94fcef2295b, user_id=13bcfaa22f2747e2be81e1238e164800, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:08.051758 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db71494-cb8f-45ed-932d-60cf996536d6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:08.064207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1217/2452] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:08 2022] PUT /identity/v3/projects/5e4aa3d2826a4a628007e94fcef2295b/users/13bcfaa22f2747e2be81e1238e164800/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:08.076455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475f6bb-14c9-4da2-b514-1a06ca02ac7e None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:08.078173 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475f6bb-14c9-4da2-b514-1a06ca02ac7e None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:08.078173 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475f6bb-14c9-4da2-b514-1a06ca02ac7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:08.111059 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-3475f6bb-14c9-4da2-b514-1a06ca02ac7e None None] MFA Rules not processed for user `13bcfaa22f2747e2be81e1238e164800`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:08.113287 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-3475f6bb-14c9-4da2-b514-1a06ca02ac7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:08.201519 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1236/2453] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:08 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:08.219196 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db71494-cb8f-45ed-932d-60cf996536d6 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:08.221599 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9db71494-cb8f-45ed-932d-60cf996536d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:08.287876 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db71494-cb8f-45ed-932d-60cf996536d6 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:08.292100 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ce3b9-3bb8-40d5-b866-bd9bcfe01788 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:08.292851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ce3b9-3bb8-40d5-b866-bd9bcfe01788 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:08.293029 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ce3b9-3bb8-40d5-b866-bd9bcfe01788 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:08.293304 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ce3b9-3bb8-40d5-b866-bd9bcfe01788 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:08.303702 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ce3b9-3bb8-40d5-b866-bd9bcfe01788 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:08.305151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ce3b9-3bb8-40d5-b866-bd9bcfe01788 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:08.317089 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1218/2454] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:08 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:20:08.708671 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475f6bb-14c9-4da2-b514-1a06ca02ac7e None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:08.718822 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475f6bb-14c9-4da2-b514-1a06ca02ac7e None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:08.723911 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07426d4-da45-404e-a7f5-89ceb0109034 tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13bcfaa22f2747e2be81e1238e164800', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e4aa3d2826a4a628007e94fcef2295b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:08.724603 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07426d4-da45-404e-a7f5-89ceb0109034 tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:08.724899 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07426d4-da45-404e-a7f5-89ceb0109034 tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:08.725178 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07426d4-da45-404e-a7f5-89ceb0109034 tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:08.725961 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07426d4-da45-404e-a7f5-89ceb0109034 tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:20:08.726324 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07426d4-da45-404e-a7f5-89ceb0109034 tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:08.728028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07426d4-da45-404e-a7f5-89ceb0109034 tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:08.730794 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:08.743567 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1237/2455] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:20:08.747321 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ce3b9-3bb8-40d5-b866-bd9bcfe01788 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:08.757605 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ce3b9-3bb8-40d5-b866-bd9bcfe01788 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:08.762660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc77f49-ec75-4b2a-8321-24b85dd3c0be tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13bcfaa22f2747e2be81e1238e164800', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e4aa3d2826a4a628007e94fcef2295b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:08.763326 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc77f49-ec75-4b2a-8321-24b85dd3c0be tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:08.763617 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc77f49-ec75-4b2a-8321-24b85dd3c0be tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:08.763906 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc77f49-ec75-4b2a-8321-24b85dd3c0be tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:08.764620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc77f49-ec75-4b2a-8321-24b85dd3c0be tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:08.767015 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc77f49-ec75-4b2a-8321-24b85dd3c0be tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:08.801527 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1219/2456] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:20:08 2022] POST /identity/v3/projects => generated 353 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:20:08.808015 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07426d4-da45-404e-a7f5-89ceb0109034 tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:08.820827 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07426d4-da45-404e-a7f5-89ceb0109034 tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:08.897822 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bda045-eb4a-496f-86f3-ac2e7f01feda tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13bcfaa22f2747e2be81e1238e164800', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e4aa3d2826a4a628007e94fcef2295b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:08.898522 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bda045-eb4a-496f-86f3-ac2e7f01feda tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:08.898865 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bda045-eb4a-496f-86f3-ac2e7f01feda tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:08.899152 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bda045-eb4a-496f-86f3-ac2e7f01feda tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:08.899909 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bda045-eb4a-496f-86f3-ac2e7f01feda tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:20:08.900249 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bda045-eb4a-496f-86f3-ac2e7f01feda tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:08.902052 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bda045-eb4a-496f-86f3-ac2e7f01feda tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:08.918428 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1238/2457] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:20:08.961367 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc77f49-ec75-4b2a-8321-24b85dd3c0be tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:08.973291 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc77f49-ec75-4b2a-8321-24b85dd3c0be tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:08.978214 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704f40cf-6b1a-40d8-9942-1d1aaa5c84c4 tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13bcfaa22f2747e2be81e1238e164800', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e4aa3d2826a4a628007e94fcef2295b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:08.979093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f40cf-6b1a-40d8-9942-1d1aaa5c84c4 tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:08.979517 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f40cf-6b1a-40d8-9942-1d1aaa5c84c4 tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:08.979709 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f40cf-6b1a-40d8-9942-1d1aaa5c84c4 tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] PATH_INFO: `/v3/projects/d37f25da6e664484a7d5fd2bf3dfc025` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:08.980889 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704f40cf-6b1a-40d8-9942-1d1aaa5c84c4 tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d37f25da6e664484a7d5fd2bf3dfc025)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:08.982898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704f40cf-6b1a-40d8-9942-1d1aaa5c84c4 tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:09.047982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-704f40cf-6b1a-40d8-9942-1d1aaa5c84c4 tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd37f25da6e664484a7d5fd2bf3dfc025', 'request_id': 'req-704f40cf-6b1a-40d8-9942-1d1aaa5c84c4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:09.053898 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-704f40cf-6b1a-40d8-9942-1d1aaa5c84c4 tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd37f25da6e664484a7d5fd2bf3dfc025', 'request_id': 'req-704f40cf-6b1a-40d8-9942-1d1aaa5c84c4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:09.066147 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-704f40cf-6b1a-40d8-9942-1d1aaa5c84c4 tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd37f25da6e664484a7d5fd2bf3dfc025', 'request_id': 'req-704f40cf-6b1a-40d8-9942-1d1aaa5c84c4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:09.092124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-704f40cf-6b1a-40d8-9942-1d1aaa5c84c4 tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] The token cache is being invalidate because project d37f25da6e664484a7d5fd2bf3dfc025 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:20:09.092553 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-704f40cf-6b1a-40d8-9942-1d1aaa5c84c4 tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:09.094256 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-704f40cf-6b1a-40d8-9942-1d1aaa5c84c4 tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:09.096367 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1220/2458] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:20:08 2022] DELETE /identity/v3/projects/d37f25da6e664484a7d5fd2bf3dfc025 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:09.346785 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bda045-eb4a-496f-86f3-ac2e7f01feda tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:09.348998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-98bda045-eb4a-496f-86f3-ac2e7f01feda tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:09.491201 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bda045-eb4a-496f-86f3-ac2e7f01feda tempest-VolumeQuotaClassesTest-366468713 tempest-VolumeQuotaClassesTest-366468713-project-admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:09.597855 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b05a5-9eac-4944-b226-109829b131f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:09.598836 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b05a5-9eac-4944-b226-109829b131f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:09.599200 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b05a5-9eac-4944-b226-109829b131f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:09.599553 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b05a5-9eac-4944-b226-109829b131f8 admin admin] PATH_INFO: `/v3/users/af15226f06334295bc056bf7490d9314` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:09.601227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b05a5-9eac-4944-b226-109829b131f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af15226f06334295bc056bf7490d9314)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:09.603568 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b05a5-9eac-4944-b226-109829b131f8 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:09.683162 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:09.694311 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7a1b05a5-9eac-4944-b226-109829b131f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af15226f06334295bc056bf7490d9314'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:09.705188 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7a1b05a5-9eac-4944-b226-109829b131f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af15226f06334295bc056bf7490d9314'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:09.721137 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7a1b05a5-9eac-4944-b226-109829b131f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af15226f06334295bc056bf7490d9314'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:09.722238 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7a1b05a5-9eac-4944-b226-109829b131f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af15226f06334295bc056bf7490d9314'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:09.731727 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7a1b05a5-9eac-4944-b226-109829b131f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af15226f06334295bc056bf7490d9314'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:09.743864 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1239/2459] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:20:09 2022] DELETE /identity/v3/users/af15226f06334295bc056bf7490d9314 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:09.752473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2f425-175c-4dbe-a3be-7a8697746ee3 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:09.752473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2f425-175c-4dbe-a3be-7a8697746ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:09.752473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2f425-175c-4dbe-a3be-7a8697746ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:09.773220 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-cdd2f425-175c-4dbe-a3be-7a8697746ee3 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:09.775123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-cdd2f425-175c-4dbe-a3be-7a8697746ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:09.788868 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:09.874841 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1221/2460] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:09.881962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b05a5-9eac-4944-b226-109829b131f8 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:09.883593 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-7a1b05a5-9eac-4944-b226-109829b131f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:09.982916 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b05a5-9eac-4944-b226-109829b131f8 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:09.990664 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af793f82-6c8d-419d-804e-27e07c7b183b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:09.992292 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af793f82-6c8d-419d-804e-27e07c7b183b service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:09.994051 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af793f82-6c8d-419d-804e-27e07c7b183b service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:09.994051 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af793f82-6c8d-419d-804e-27e07c7b183b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:10.007833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af793f82-6c8d-419d-804e-27e07c7b183b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:10.009779 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af793f82-6c8d-419d-804e-27e07c7b183b service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:10.023703 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1240/2461] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:20:10.320767 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2f425-175c-4dbe-a3be-7a8697746ee3 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:10.322654 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-cdd2f425-175c-4dbe-a3be-7a8697746ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:10.348385 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2f425-175c-4dbe-a3be-7a8697746ee3 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:10.354643 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd969bf4-20ef-492b-aa11-cac1163d286b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:10.356741 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd969bf4-20ef-492b-aa11-cac1163d286b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:10.357422 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd969bf4-20ef-492b-aa11-cac1163d286b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:10.357886 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd969bf4-20ef-492b-aa11-cac1163d286b admin admin] PATH_INFO: `/v3/projects/b6b6f0e060c1404798ddf9def43ee055` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:10.361821 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd969bf4-20ef-492b-aa11-cac1163d286b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6b6f0e060c1404798ddf9def43ee055)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:10.364920 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd969bf4-20ef-492b-aa11-cac1163d286b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:10.418087 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-cd969bf4-20ef-492b-aa11-cac1163d286b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6b6f0e060c1404798ddf9def43ee055', 'request_id': 'req-cd969bf4-20ef-492b-aa11-cac1163d286b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:10.425333 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-cd969bf4-20ef-492b-aa11-cac1163d286b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6b6f0e060c1404798ddf9def43ee055', 'request_id': 'req-cd969bf4-20ef-492b-aa11-cac1163d286b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:10.435250 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-cd969bf4-20ef-492b-aa11-cac1163d286b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6b6f0e060c1404798ddf9def43ee055', 'request_id': 'req-cd969bf4-20ef-492b-aa11-cac1163d286b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:10.451517 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-cd969bf4-20ef-492b-aa11-cac1163d286b admin admin] The token cache is being invalidate because project b6b6f0e060c1404798ddf9def43ee055 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:20:10.451894 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-cd969bf4-20ef-492b-aa11-cac1163d286b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:10.453954 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-cd969bf4-20ef-492b-aa11-cac1163d286b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:10.457047 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1222/2462] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:20:10 2022] DELETE /identity/v3/projects/b6b6f0e060c1404798ddf9def43ee055 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:10.462590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af793f82-6c8d-419d-804e-27e07c7b183b service neutron] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:10.474594 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af793f82-6c8d-419d-804e-27e07c7b183b service neutron] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:10.545053 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9f642-3db3-451e-9827-578a7008c051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:10.547056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9f642-3db3-451e-9827-578a7008c051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:10.547498 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9f642-3db3-451e-9827-578a7008c051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:10.547929 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9f642-3db3-451e-9827-578a7008c051 admin admin] PATH_INFO: `/v3/users/13bcfaa22f2747e2be81e1238e164800` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:10.549455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa9f642-3db3-451e-9827-578a7008c051 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13bcfaa22f2747e2be81e1238e164800)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:10.552819 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa9f642-3db3-451e-9827-578a7008c051 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:10.613982 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-1fa9f642-3db3-451e-9827-578a7008c051 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13bcfaa22f2747e2be81e1238e164800'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:10.627035 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-1fa9f642-3db3-451e-9827-578a7008c051 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13bcfaa22f2747e2be81e1238e164800'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:10.633156 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:10.644335 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-1fa9f642-3db3-451e-9827-578a7008c051 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13bcfaa22f2747e2be81e1238e164800'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:10.645085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-1fa9f642-3db3-451e-9827-578a7008c051 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13bcfaa22f2747e2be81e1238e164800'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:10.652896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-1fa9f642-3db3-451e-9827-578a7008c051 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13bcfaa22f2747e2be81e1238e164800'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:10.661499 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1241/2463] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:20:10 2022] DELETE /identity/v3/users/13bcfaa22f2747e2be81e1238e164800 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:10.990733 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd969bf4-20ef-492b-aa11-cac1163d286b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:10.992989 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-cd969bf4-20ef-492b-aa11-cac1163d286b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:11.010662 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd969bf4-20ef-492b-aa11-cac1163d286b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:11.088573 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda12d7-78d9-47d7-80eb-5d29d29e57c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:11.089622 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda12d7-78d9-47d7-80eb-5d29d29e57c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:11.090013 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda12d7-78d9-47d7-80eb-5d29d29e57c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:11.090362 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda12d7-78d9-47d7-80eb-5d29d29e57c7 admin admin] PATH_INFO: `/v3/projects/5e4aa3d2826a4a628007e94fcef2295b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:11.092019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda12d7-78d9-47d7-80eb-5d29d29e57c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e4aa3d2826a4a628007e94fcef2295b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:11.094102 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda12d7-78d9-47d7-80eb-5d29d29e57c7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:11.146299 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:11.158234 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9bda12d7-78d9-47d7-80eb-5d29d29e57c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e4aa3d2826a4a628007e94fcef2295b', 'request_id': 'req-9bda12d7-78d9-47d7-80eb-5d29d29e57c7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:11.166274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9bda12d7-78d9-47d7-80eb-5d29d29e57c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e4aa3d2826a4a628007e94fcef2295b', 'request_id': 'req-9bda12d7-78d9-47d7-80eb-5d29d29e57c7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:11.177748 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9bda12d7-78d9-47d7-80eb-5d29d29e57c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e4aa3d2826a4a628007e94fcef2295b', 'request_id': 'req-9bda12d7-78d9-47d7-80eb-5d29d29e57c7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:11.191431 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9bda12d7-78d9-47d7-80eb-5d29d29e57c7 admin admin] The token cache is being invalidate because project 5e4aa3d2826a4a628007e94fcef2295b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:20:11.191994 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9bda12d7-78d9-47d7-80eb-5d29d29e57c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:11.192698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9bda12d7-78d9-47d7-80eb-5d29d29e57c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:11.194383 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1223/2464] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:20:10 2022] DELETE /identity/v3/projects/5e4aa3d2826a4a628007e94fcef2295b => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:11.199665 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8efff3-20fe-41cf-9632-74c7c7316cce None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:11.200213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8efff3-20fe-41cf-9632-74c7c7316cce None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:11.200793 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8efff3-20fe-41cf-9632-74c7c7316cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:11.225557 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-9d8efff3-20fe-41cf-9632-74c7c7316cce None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:11.227046 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-9d8efff3-20fe-41cf-9632-74c7c7316cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:11.307180 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1242/2465] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:20:11.316980 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b602bcb-a864-44ab-a21c-d96bdd2ca6a8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:11.317338 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b602bcb-a864-44ab-a21c-d96bdd2ca6a8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:11.317689 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b602bcb-a864-44ab-a21c-d96bdd2ca6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:11.338284 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0b602bcb-a864-44ab-a21c-d96bdd2ca6a8 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:11.339559 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0b602bcb-a864-44ab-a21c-d96bdd2ca6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:11.351871 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1224/2466] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:11.353194 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8efff3-20fe-41cf-9632-74c7c7316cce None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:11.366051 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8efff3-20fe-41cf-9632-74c7c7316cce None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:11.371206 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb97fce2-e587-4ec9-8bca-2d8fed9beb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:11.371893 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97fce2-e587-4ec9-8bca-2d8fed9beb70 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:11.372190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97fce2-e587-4ec9-8bca-2d8fed9beb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:11.372475 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97fce2-e587-4ec9-8bca-2d8fed9beb70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:11.373246 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb97fce2-e587-4ec9-8bca-2d8fed9beb70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:20:11.373555 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb97fce2-e587-4ec9-8bca-2d8fed9beb70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:11.376408 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb97fce2-e587-4ec9-8bca-2d8fed9beb70 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:11.389794 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1243/2467] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:20:11.394408 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b602bcb-a864-44ab-a21c-d96bdd2ca6a8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:11.404065 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b602bcb-a864-44ab-a21c-d96bdd2ca6a8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:11.409438 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c178822-4a16-450c-99f4-e19585576871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:11.410130 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c178822-4a16-450c-99f4-e19585576871 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:11.410552 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c178822-4a16-450c-99f4-e19585576871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:11.410935 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c178822-4a16-450c-99f4-e19585576871 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:11.411726 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c178822-4a16-450c-99f4-e19585576871 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:11.414092 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c178822-4a16-450c-99f4-e19585576871 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:11.445089 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1225/2468] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:20:11 2022] POST /identity/v3/projects => generated 393 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:11.450254 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb97fce2-e587-4ec9-8bca-2d8fed9beb70 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:11.463373 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb97fce2-e587-4ec9-8bca-2d8fed9beb70 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:11.537669 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4433a-67ed-4611-a93d-38d7bb3c156f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:11.538725 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4433a-67ed-4611-a93d-38d7bb3c156f admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:11.538725 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4433a-67ed-4611-a93d-38d7bb3c156f admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:11.539087 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4433a-67ed-4611-a93d-38d7bb3c156f admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:11.539865 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f4433a-67ed-4611-a93d-38d7bb3c156f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:11.541840 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f4433a-67ed-4611-a93d-38d7bb3c156f admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:11.568462 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1244/2469] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:20:11 2022] POST /identity/v3/users => generated 297 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:11.573177 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c178822-4a16-450c-99f4-e19585576871 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:11.585329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c178822-4a16-450c-99f4-e19585576871 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:11.592024 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5efe2-5c0e-4a38-9064-43875720e2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:11.592715 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5efe2-5c0e-4a38-9064-43875720e2e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:11.593021 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5efe2-5c0e-4a38-9064-43875720e2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:11.593301 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5efe2-5c0e-4a38-9064-43875720e2e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:11.594146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5efe2-5c0e-4a38-9064-43875720e2e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:11.595929 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5efe2-5c0e-4a38-9064-43875720e2e3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:11.611204 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1226/2470] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:11.616112 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4433a-67ed-4611-a93d-38d7bb3c156f admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:11.628736 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4433a-67ed-4611-a93d-38d7bb3c156f admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:11.635050 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c64fd-5956-44b7-8172-eb2f353b76bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:11.635732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c64fd-5956-44b7-8172-eb2f353b76bb admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:11.636020 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c64fd-5956-44b7-8172-eb2f353b76bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:11.636296 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c64fd-5956-44b7-8172-eb2f353b76bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:11.637033 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c64fd-5956-44b7-8172-eb2f353b76bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:11.638712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c64fd-5956-44b7-8172-eb2f353b76bb admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:11.652568 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1245/2471] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:20:11.658865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5efe2-5c0e-4a38-9064-43875720e2e3 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:11.670472 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5efe2-5c0e-4a38-9064-43875720e2e3 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:11.677156 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93334bd0-ee4a-4ffa-bbd9-3bad9262143c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:11.678117 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93334bd0-ee4a-4ffa-bbd9-3bad9262143c admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:11.678416 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93334bd0-ee4a-4ffa-bbd9-3bad9262143c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:11.678699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93334bd0-ee4a-4ffa-bbd9-3bad9262143c admin admin] PATH_INFO: `/v3/projects/bbcc1cf5dfc740a09a46b489f5150cb9/users/40d3350585614df4b0ec43523edb0a2e/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:11.700027 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93334bd0-ee4a-4ffa-bbd9-3bad9262143c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbcc1cf5dfc740a09a46b489f5150cb9, user_id=40d3350585614df4b0ec43523edb0a2e, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:11.703441 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93334bd0-ee4a-4ffa-bbd9-3bad9262143c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:11.718938 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1227/2472] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:11 2022] PUT /identity/v3/projects/bbcc1cf5dfc740a09a46b489f5150cb9/users/40d3350585614df4b0ec43523edb0a2e/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:11.729300 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1a148-7b18-456d-b4de-49735b254da9 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:11.729594 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1a148-7b18-456d-b4de-49735b254da9 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:11.729876 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1a148-7b18-456d-b4de-49735b254da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:11.760263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-c3a1a148-7b18-456d-b4de-49735b254da9 None None] MFA Rules not processed for user `40d3350585614df4b0ec43523edb0a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:11.761610 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-c3a1a148-7b18-456d-b4de-49735b254da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:11.807467 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:11.850403 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1246/2473] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:20:11.854549 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93334bd0-ee4a-4ffa-bbd9-3bad9262143c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:11.942694 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93334bd0-ee4a-4ffa-bbd9-3bad9262143c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:11.948095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2213428-7e4a-4d79-8215-c18decdb10ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:11.948774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2213428-7e4a-4d79-8215-c18decdb10ea admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:11.949060 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2213428-7e4a-4d79-8215-c18decdb10ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:11.949333 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2213428-7e4a-4d79-8215-c18decdb10ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:11.950077 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2213428-7e4a-4d79-8215-c18decdb10ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:11.952079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2213428-7e4a-4d79-8215-c18decdb10ea admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:11.982508 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1228/2474] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:20:11 2022] POST /identity/v3/projects => generated 393 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:20:11.988055 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1a148-7b18-456d-b4de-49735b254da9 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:12.000604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1a148-7b18-456d-b4de-49735b254da9 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:12.073729 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec696366-321d-4b09-b927-878cfca16303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:12.074476 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec696366-321d-4b09-b927-878cfca16303 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:12.074762 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec696366-321d-4b09-b927-878cfca16303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:12.075075 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec696366-321d-4b09-b927-878cfca16303 admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:12.075821 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec696366-321d-4b09-b927-878cfca16303 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:12.078176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec696366-321d-4b09-b927-878cfca16303 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:12.103101 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1247/2475] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:20:11 2022] POST /identity/v3/users => generated 303 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:12.107478 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2213428-7e4a-4d79-8215-c18decdb10ea admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:12.123934 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2213428-7e4a-4d79-8215-c18decdb10ea admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:12.132417 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007115d7-f66e-4dc0-bf66-cf0692902202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:12.133298 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007115d7-f66e-4dc0-bf66-cf0692902202 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:12.133685 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007115d7-f66e-4dc0-bf66-cf0692902202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:12.134553 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007115d7-f66e-4dc0-bf66-cf0692902202 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:12.135484 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007115d7-f66e-4dc0-bf66-cf0692902202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:12.137829 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007115d7-f66e-4dc0-bf66-cf0692902202 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:12.153212 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1229/2476] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:12.158917 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec696366-321d-4b09-b927-878cfca16303 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:12.169813 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec696366-321d-4b09-b927-878cfca16303 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:12.177833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d3458-61c3-4a2f-83de-954f32f1cc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:12.178986 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d3458-61c3-4a2f-83de-954f32f1cc80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:12.179368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d3458-61c3-4a2f-83de-954f32f1cc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:12.179858 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d3458-61c3-4a2f-83de-954f32f1cc80 admin admin] PATH_INFO: `/v3/projects/8982bd6be4f74ba092ac989094ae2f27/users/571e5baa32454a2c9caf1477bca84c78/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:12.197481 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d3458-61c3-4a2f-83de-954f32f1cc80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8982bd6be4f74ba092ac989094ae2f27, user_id=571e5baa32454a2c9caf1477bca84c78, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:12.200135 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d3458-61c3-4a2f-83de-954f32f1cc80 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:12.217081 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1248/2477] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:12 2022] PUT /identity/v3/projects/8982bd6be4f74ba092ac989094ae2f27/users/571e5baa32454a2c9caf1477bca84c78/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:20:12.222263 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007115d7-f66e-4dc0-bf66-cf0692902202 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:12.234768 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007115d7-f66e-4dc0-bf66-cf0692902202 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:12.271995 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:12.302084 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b0280-a65e-41fb-bb2f-5140a121f452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:12.302768 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b0280-a65e-41fb-bb2f-5140a121f452 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:12.303051 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b0280-a65e-41fb-bb2f-5140a121f452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:12.303331 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b0280-a65e-41fb-bb2f-5140a121f452 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:12.304063 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70b0280-a65e-41fb-bb2f-5140a121f452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:12.305680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70b0280-a65e-41fb-bb2f-5140a121f452 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:12.318861 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1230/2478] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:12.323176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d3458-61c3-4a2f-83de-954f32f1cc80 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:12.335195 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d3458-61c3-4a2f-83de-954f32f1cc80 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:12.340660 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f06709a-0c62-4836-8fc1-b0c3596a8cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:12.341600 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06709a-0c62-4836-8fc1-b0c3596a8cad admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:12.341836 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06709a-0c62-4836-8fc1-b0c3596a8cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:12.342313 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06709a-0c62-4836-8fc1-b0c3596a8cad admin admin] PATH_INFO: `/v3/projects/8982bd6be4f74ba092ac989094ae2f27/users/571e5baa32454a2c9caf1477bca84c78/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:12.345522 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f06709a-0c62-4836-8fc1-b0c3596a8cad admin admin] RBAC: Authorizing `identity:create_grant(project_id=8982bd6be4f74ba092ac989094ae2f27, user_id=571e5baa32454a2c9caf1477bca84c78, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:12.348258 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f06709a-0c62-4836-8fc1-b0c3596a8cad admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:12.360761 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1249/2479] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:12 2022] PUT /identity/v3/projects/8982bd6be4f74ba092ac989094ae2f27/users/571e5baa32454a2c9caf1477bca84c78/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:12.372432 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a5619-08c1-4bbc-a331-b75e1b7c8baa None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:12.372745 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a5619-08c1-4bbc-a331-b75e1b7c8baa None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:12.373038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a5619-08c1-4bbc-a331-b75e1b7c8baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:12.403112 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-eb9a5619-08c1-4bbc-a331-b75e1b7c8baa None None] MFA Rules not processed for user `571e5baa32454a2c9caf1477bca84c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:12.404356 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-eb9a5619-08c1-4bbc-a331-b75e1b7c8baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:12.496614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1231/2480] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:12 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:12.511756 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f06709a-0c62-4836-8fc1-b0c3596a8cad admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:12.514117 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-4f06709a-0c62-4836-8fc1-b0c3596a8cad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:12.583781 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f06709a-0c62-4836-8fc1-b0c3596a8cad admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:12.588237 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd56c6-b224-427c-9093-3fd38e4dd3b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:12.588960 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd56c6-b224-427c-9093-3fd38e4dd3b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:12.589246 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd56c6-b224-427c-9093-3fd38e4dd3b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:12.589546 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd56c6-b224-427c-9093-3fd38e4dd3b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:12.600242 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dd56c6-b224-427c-9093-3fd38e4dd3b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:12.601845 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dd56c6-b224-427c-9093-3fd38e4dd3b8 service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:12.613272 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1250/2481] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:20:12.667629 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a5619-08c1-4bbc-a331-b75e1b7c8baa None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:12.669573 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-eb9a5619-08c1-4bbc-a331-b75e1b7c8baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:12.683615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a5619-08c1-4bbc-a331-b75e1b7c8baa None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:12.688911 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb87a4-b0c8-4396-bd2e-c5157b56005b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:12.689859 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb87a4-b0c8-4396-bd2e-c5157b56005b service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:12.690320 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb87a4-b0c8-4396-bd2e-c5157b56005b service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:12.690610 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb87a4-b0c8-4396-bd2e-c5157b56005b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:12.756717 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cb87a4-b0c8-4396-bd2e-c5157b56005b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:12.758127 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cb87a4-b0c8-4396-bd2e-c5157b56005b service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:12.768877 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1232/2482] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:20:15.533745 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd56c6-b224-427c-9093-3fd38e4dd3b8 service cinder] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:15.535388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-39dd56c6-b224-427c-9093-3fd38e4dd3b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:15.574836 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:15.612342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd56c6-b224-427c-9093-3fd38e4dd3b8 service cinder] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:15.618836 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce267d49-52c9-4f0e-98b8-4a654d4aec44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:15.619917 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce267d49-52c9-4f0e-98b8-4a654d4aec44 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:15.620402 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce267d49-52c9-4f0e-98b8-4a654d4aec44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:15.620870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce267d49-52c9-4f0e-98b8-4a654d4aec44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:15.624481 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-ce267d49-52c9-4f0e-98b8-4a654d4aec44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:15.701741 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce267d49-52c9-4f0e-98b8-4a654d4aec44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:15.704342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce267d49-52c9-4f0e-98b8-4a654d4aec44 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:15.716473 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1251/2483] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:20:16.210094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb87a4-b0c8-4396-bd2e-c5157b56005b service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:16.221860 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb87a4-b0c8-4396-bd2e-c5157b56005b service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:16.226758 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3878da7-563d-464f-8789-e5f19c95823d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:16.227643 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3878da7-563d-464f-8789-e5f19c95823d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:16.227931 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3878da7-563d-464f-8789-e5f19c95823d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:16.228208 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3878da7-563d-464f-8789-e5f19c95823d admin admin] PATH_INFO: `/v3/users/40d3350585614df4b0ec43523edb0a2e` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:16.229444 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3878da7-563d-464f-8789-e5f19c95823d admin admin] RBAC: Authorizing `identity:delete_user(user_id=40d3350585614df4b0ec43523edb0a2e)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:16.232158 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3878da7-563d-464f-8789-e5f19c95823d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:16.291951 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e3878da7-563d-464f-8789-e5f19c95823d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40d3350585614df4b0ec43523edb0a2e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:16.297488 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:16.308154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e3878da7-563d-464f-8789-e5f19c95823d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40d3350585614df4b0ec43523edb0a2e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:16.316397 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e3878da7-563d-464f-8789-e5f19c95823d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40d3350585614df4b0ec43523edb0a2e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:16.324806 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e3878da7-563d-464f-8789-e5f19c95823d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40d3350585614df4b0ec43523edb0a2e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:16.325552 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e3878da7-563d-464f-8789-e5f19c95823d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40d3350585614df4b0ec43523edb0a2e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:16.332773 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1233/2484] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:20:16 2022] DELETE /identity/v3/users/40d3350585614df4b0ec43523edb0a2e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:16.337516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7049c-00cf-4025-9dae-9a7763b812ff None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:16.337814 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7049c-00cf-4025-9dae-9a7763b812ff None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:16.338123 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7049c-00cf-4025-9dae-9a7763b812ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:16.356530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-73a7049c-00cf-4025-9dae-9a7763b812ff None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:16.359086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-73a7049c-00cf-4025-9dae-9a7763b812ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:16.433997 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1252/2485] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:16.445723 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3878da7-563d-464f-8789-e5f19c95823d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:16.448360 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e3878da7-563d-464f-8789-e5f19c95823d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:16.517461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3878da7-563d-464f-8789-e5f19c95823d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:16.521352 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfee445-3342-444e-b6c0-bc2db93a64ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:16.522210 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfee445-3342-444e-b6c0-bc2db93a64ab service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:16.522412 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfee445-3342-444e-b6c0-bc2db93a64ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:16.522695 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfee445-3342-444e-b6c0-bc2db93a64ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:16.534544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfee445-3342-444e-b6c0-bc2db93a64ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:16.536572 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfee445-3342-444e-b6c0-bc2db93a64ab service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:16.548044 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1234/2486] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:20:17.302528 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7049c-00cf-4025-9dae-9a7763b812ff None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:17.304840 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-73a7049c-00cf-4025-9dae-9a7763b812ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:17.320025 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7049c-00cf-4025-9dae-9a7763b812ff None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:17.325999 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd11b44-dd99-4a08-a21e-52e6210ffe41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:17.327023 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd11b44-dd99-4a08-a21e-52e6210ffe41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:17.327231 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd11b44-dd99-4a08-a21e-52e6210ffe41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:17.328053 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd11b44-dd99-4a08-a21e-52e6210ffe41 admin admin] PATH_INFO: `/v3/projects/bbcc1cf5dfc740a09a46b489f5150cb9` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:17.329803 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd11b44-dd99-4a08-a21e-52e6210ffe41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbcc1cf5dfc740a09a46b489f5150cb9)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:17.331919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd11b44-dd99-4a08-a21e-52e6210ffe41 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:17.401137 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-bdd11b44-dd99-4a08-a21e-52e6210ffe41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbcc1cf5dfc740a09a46b489f5150cb9', 'request_id': 'req-bdd11b44-dd99-4a08-a21e-52e6210ffe41'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:17.409497 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-bdd11b44-dd99-4a08-a21e-52e6210ffe41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbcc1cf5dfc740a09a46b489f5150cb9', 'request_id': 'req-bdd11b44-dd99-4a08-a21e-52e6210ffe41'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:17.418350 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-bdd11b44-dd99-4a08-a21e-52e6210ffe41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbcc1cf5dfc740a09a46b489f5150cb9', 'request_id': 'req-bdd11b44-dd99-4a08-a21e-52e6210ffe41'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:17.432357 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-bdd11b44-dd99-4a08-a21e-52e6210ffe41 admin admin] The token cache is being invalidate because project bbcc1cf5dfc740a09a46b489f5150cb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:20:17.432786 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-bdd11b44-dd99-4a08-a21e-52e6210ffe41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:17.433366 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-bdd11b44-dd99-4a08-a21e-52e6210ffe41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:17.434984 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1253/2487] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:20:17 2022] DELETE /identity/v3/projects/bbcc1cf5dfc740a09a46b489f5150cb9 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:17.440144 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfee445-3342-444e-b6c0-bc2db93a64ab service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:17.450260 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfee445-3342-444e-b6c0-bc2db93a64ab service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:17.522317 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7efbd3-22ae-4f9c-8348-606e38981587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:17.523196 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7efbd3-22ae-4f9c-8348-606e38981587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:17.523477 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7efbd3-22ae-4f9c-8348-606e38981587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:17.523753 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7efbd3-22ae-4f9c-8348-606e38981587 admin admin] PATH_INFO: `/v3/users/571e5baa32454a2c9caf1477bca84c78` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:17.524958 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7efbd3-22ae-4f9c-8348-606e38981587 admin admin] RBAC: Authorizing `identity:delete_user(user_id=571e5baa32454a2c9caf1477bca84c78)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:17.526921 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7efbd3-22ae-4f9c-8348-606e38981587 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:17.586487 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5a7efbd3-22ae-4f9c-8348-606e38981587 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '571e5baa32454a2c9caf1477bca84c78'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:17.604706 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5a7efbd3-22ae-4f9c-8348-606e38981587 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '571e5baa32454a2c9caf1477bca84c78'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:17.620793 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5a7efbd3-22ae-4f9c-8348-606e38981587 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '571e5baa32454a2c9caf1477bca84c78'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:17.625186 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:17.634511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5a7efbd3-22ae-4f9c-8348-606e38981587 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '571e5baa32454a2c9caf1477bca84c78'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:17.636334 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5a7efbd3-22ae-4f9c-8348-606e38981587 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '571e5baa32454a2c9caf1477bca84c78'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:17.647047 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1235/2488] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:20:17 2022] DELETE /identity/v3/users/571e5baa32454a2c9caf1477bca84c78 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:20:18.082890 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd11b44-dd99-4a08-a21e-52e6210ffe41 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:18.084960 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-bdd11b44-dd99-4a08-a21e-52e6210ffe41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:18.094948 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:18.112513 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd11b44-dd99-4a08-a21e-52e6210ffe41 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:18.200545 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18727c59-246d-464a-aa1e-0b2f1d9b6d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:18.201380 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18727c59-246d-464a-aa1e-0b2f1d9b6d99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:18.201667 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18727c59-246d-464a-aa1e-0b2f1d9b6d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:18.202479 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18727c59-246d-464a-aa1e-0b2f1d9b6d99 admin admin] PATH_INFO: `/v3/projects/8982bd6be4f74ba092ac989094ae2f27` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:18.203745 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18727c59-246d-464a-aa1e-0b2f1d9b6d99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8982bd6be4f74ba092ac989094ae2f27)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:18.205699 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18727c59-246d-464a-aa1e-0b2f1d9b6d99 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:18.263437 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-18727c59-246d-464a-aa1e-0b2f1d9b6d99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8982bd6be4f74ba092ac989094ae2f27', 'request_id': 'req-18727c59-246d-464a-aa1e-0b2f1d9b6d99'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:18.270707 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-18727c59-246d-464a-aa1e-0b2f1d9b6d99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8982bd6be4f74ba092ac989094ae2f27', 'request_id': 'req-18727c59-246d-464a-aa1e-0b2f1d9b6d99'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:18.279052 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-18727c59-246d-464a-aa1e-0b2f1d9b6d99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8982bd6be4f74ba092ac989094ae2f27', 'request_id': 'req-18727c59-246d-464a-aa1e-0b2f1d9b6d99'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:18.292251 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-18727c59-246d-464a-aa1e-0b2f1d9b6d99 admin admin] The token cache is being invalidate because project 8982bd6be4f74ba092ac989094ae2f27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:20:18.292637 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-18727c59-246d-464a-aa1e-0b2f1d9b6d99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:18.293674 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-18727c59-246d-464a-aa1e-0b2f1d9b6d99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:18.295614 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1254/2489] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:20:18 2022] DELETE /identity/v3/projects/8982bd6be4f74ba092ac989094ae2f27 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:18.302119 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c429b-02b6-4ccd-9b55-6bc255edea0a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:18.302413 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c429b-02b6-4ccd-9b55-6bc255edea0a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:18.302691 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c429b-02b6-4ccd-9b55-6bc255edea0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:18.322092 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-bc9c429b-02b6-4ccd-9b55-6bc255edea0a None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:18.323608 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bc9c429b-02b6-4ccd-9b55-6bc255edea0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:18.399577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1236/2490] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:20:18.408186 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbc17b-a567-43e0-ac58-52ac7fc08c6b None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:18.408724 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbc17b-a567-43e0-ac58-52ac7fc08c6b None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:18.408851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbc17b-a567-43e0-ac58-52ac7fc08c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:18.428540 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-f7fbc17b-a567-43e0-ac58-52ac7fc08c6b None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:18.429715 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-f7fbc17b-a567-43e0-ac58-52ac7fc08c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:18.438609 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1255/2491] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:18.442896 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c429b-02b6-4ccd-9b55-6bc255edea0a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:18.453794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c429b-02b6-4ccd-9b55-6bc255edea0a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:18.458703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064dd2b-663a-4c99-8429-3c0c2e4451b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:18.459385 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064dd2b-663a-4c99-8429-3c0c2e4451b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:18.459672 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064dd2b-663a-4c99-8429-3c0c2e4451b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:18.459950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064dd2b-663a-4c99-8429-3c0c2e4451b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:18.460687 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064dd2b-663a-4c99-8429-3c0c2e4451b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:20:18.461006 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064dd2b-663a-4c99-8429-3c0c2e4451b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:18.462696 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064dd2b-663a-4c99-8429-3c0c2e4451b0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:18.478319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1237/2492] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:20:18.482574 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fbc17b-a567-43e0-ac58-52ac7fc08c6b None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:18.492790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fbc17b-a567-43e0-ac58-52ac7fc08c6b None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:18.498597 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa6117-a2a1-4ec6-804e-956fbd8542b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:18.500018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa6117-a2a1-4ec6-804e-956fbd8542b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:18.500334 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa6117-a2a1-4ec6-804e-956fbd8542b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:18.501285 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa6117-a2a1-4ec6-804e-956fbd8542b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:18.502998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa6117-a2a1-4ec6-804e-956fbd8542b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:18.506521 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa6117-a2a1-4ec6-804e-956fbd8542b8 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:18.540850 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1256/2493] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:20:18 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:18.544933 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064dd2b-663a-4c99-8429-3c0c2e4451b0 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:18.559459 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064dd2b-663a-4c99-8429-3c0c2e4451b0 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:18.668127 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0501ae-442a-4e5c-8ce2-4f837913a3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:18.669112 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0501ae-442a-4e5c-8ce2-4f837913a3be admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:18.669468 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0501ae-442a-4e5c-8ce2-4f837913a3be admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:18.669671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0501ae-442a-4e5c-8ce2-4f837913a3be admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:18.670720 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0501ae-442a-4e5c-8ce2-4f837913a3be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:18.673594 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0501ae-442a-4e5c-8ce2-4f837913a3be admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:18.706118 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1238/2494] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:20:18 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:20:18.707343 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa6117-a2a1-4ec6-804e-956fbd8542b8 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:18.718000 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa6117-a2a1-4ec6-804e-956fbd8542b8 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:18.724195 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23365e6-3680-46eb-98db-9a1df0c3d58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:18.724893 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23365e6-3680-46eb-98db-9a1df0c3d58c admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:18.725182 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23365e6-3680-46eb-98db-9a1df0c3d58c admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:18.725462 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23365e6-3680-46eb-98db-9a1df0c3d58c admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:18.726243 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23365e6-3680-46eb-98db-9a1df0c3d58c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:18.727944 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23365e6-3680-46eb-98db-9a1df0c3d58c admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:18.740282 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1257/2495] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:20:18.744803 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0501ae-442a-4e5c-8ce2-4f837913a3be admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:18.756480 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0501ae-442a-4e5c-8ce2-4f837913a3be admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:18.761774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56778e-61d5-498d-814c-e6935949ae00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:18.762581 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56778e-61d5-498d-814c-e6935949ae00 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:18.762939 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56778e-61d5-498d-814c-e6935949ae00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:18.763289 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56778e-61d5-498d-814c-e6935949ae00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:18.764111 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56778e-61d5-498d-814c-e6935949ae00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:18.767243 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56778e-61d5-498d-814c-e6935949ae00 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:18.782622 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1239/2496] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:20:18.788097 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23365e6-3680-46eb-98db-9a1df0c3d58c admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:18.799629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23365e6-3680-46eb-98db-9a1df0c3d58c admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:18.804592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ad9a7-fcd1-4427-9398-eabf16b00ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:18.805502 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ad9a7-fcd1-4427-9398-eabf16b00ef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:18.805787 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ad9a7-fcd1-4427-9398-eabf16b00ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:18.806115 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ad9a7-fcd1-4427-9398-eabf16b00ef0 admin admin] PATH_INFO: `/v3/projects/32ce91baa96b41128461ad8c72e68c27/users/f25373331d774b6db279285644773e7b/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:18.825354 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ad9a7-fcd1-4427-9398-eabf16b00ef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32ce91baa96b41128461ad8c72e68c27, user_id=f25373331d774b6db279285644773e7b, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:18.826113 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ad9a7-fcd1-4427-9398-eabf16b00ef0 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:18.842793 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1258/2497] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:18 2022] PUT /identity/v3/projects/32ce91baa96b41128461ad8c72e68c27/users/f25373331d774b6db279285644773e7b/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:18.855290 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d8ff0-5b87-4f3e-a949-9bee90c82a95 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:18.855680 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d8ff0-5b87-4f3e-a949-9bee90c82a95 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:18.855967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d8ff0-5b87-4f3e-a949-9bee90c82a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:18.897513 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b19d8ff0-5b87-4f3e-a949-9bee90c82a95 None None] MFA Rules not processed for user `f25373331d774b6db279285644773e7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:18.898744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b19d8ff0-5b87-4f3e-a949-9bee90c82a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:18.949249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:19.000233 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1240/2498] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:19.007890 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ad9a7-fcd1-4427-9398-eabf16b00ef0 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:19.077487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:19.135979 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ad9a7-fcd1-4427-9398-eabf16b00ef0 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:19.142495 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550dc745-05f1-4b11-85b1-1a00efc6f37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:19.143741 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dc745-05f1-4b11-85b1-1a00efc6f37c admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:19.144248 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dc745-05f1-4b11-85b1-1a00efc6f37c admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:19.145225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dc745-05f1-4b11-85b1-1a00efc6f37c admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:19.146243 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550dc745-05f1-4b11-85b1-1a00efc6f37c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:19.149316 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550dc745-05f1-4b11-85b1-1a00efc6f37c admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:19.183962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1259/2499] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:20:19 2022] POST /identity/v3/projects => generated 377 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:19.188715 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d8ff0-5b87-4f3e-a949-9bee90c82a95 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:19.204304 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d8ff0-5b87-4f3e-a949-9bee90c82a95 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:19.315687 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e6e004-049b-4c80-bc5e-2d6a5dd449be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:19.317092 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6e004-049b-4c80-bc5e-2d6a5dd449be admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:19.318174 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6e004-049b-4c80-bc5e-2d6a5dd449be admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:19.318647 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6e004-049b-4c80-bc5e-2d6a5dd449be admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:19.320897 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e6e004-049b-4c80-bc5e-2d6a5dd449be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:19.324992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e6e004-049b-4c80-bc5e-2d6a5dd449be admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:19.380618 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1241/2500] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:20:19 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:20:19.386516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550dc745-05f1-4b11-85b1-1a00efc6f37c admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:19.405782 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550dc745-05f1-4b11-85b1-1a00efc6f37c admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:19.414479 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b68ccb4-79b2-465d-846c-7fe9fb5970e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:19.415218 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68ccb4-79b2-465d-846c-7fe9fb5970e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:19.415512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68ccb4-79b2-465d-846c-7fe9fb5970e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:19.415789 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68ccb4-79b2-465d-846c-7fe9fb5970e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:19.416527 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b68ccb4-79b2-465d-846c-7fe9fb5970e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:19.418525 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b68ccb4-79b2-465d-846c-7fe9fb5970e4 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:19.434409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1260/2501] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:19.439319 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e6e004-049b-4c80-bc5e-2d6a5dd449be admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:19.451214 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e6e004-049b-4c80-bc5e-2d6a5dd449be admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:19.459058 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf068f9-a194-4312-b6d0-94578fc4a944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:19.459752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf068f9-a194-4312-b6d0-94578fc4a944 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:19.460085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf068f9-a194-4312-b6d0-94578fc4a944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:19.460366 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf068f9-a194-4312-b6d0-94578fc4a944 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:19.461423 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf068f9-a194-4312-b6d0-94578fc4a944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:19.464380 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf068f9-a194-4312-b6d0-94578fc4a944 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:19.477855 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1242/2502] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:19.484640 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b68ccb4-79b2-465d-846c-7fe9fb5970e4 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:19.498842 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b68ccb4-79b2-465d-846c-7fe9fb5970e4 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:19.504593 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f796f83-07d0-471b-a8e6-36e3fcd1e7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:19.505626 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f796f83-07d0-471b-a8e6-36e3fcd1e7d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:19.505775 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f796f83-07d0-471b-a8e6-36e3fcd1e7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:19.506076 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f796f83-07d0-471b-a8e6-36e3fcd1e7d1 admin admin] PATH_INFO: `/v3/projects/4ddec6e8e54842b28085b39daa338f1e/users/5627ee24f25e40be96b9f575d368d8cf/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:19.522586 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f796f83-07d0-471b-a8e6-36e3fcd1e7d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ddec6e8e54842b28085b39daa338f1e, user_id=5627ee24f25e40be96b9f575d368d8cf, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:19.525354 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f796f83-07d0-471b-a8e6-36e3fcd1e7d1 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:19.542523 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1261/2503] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:19 2022] PUT /identity/v3/projects/4ddec6e8e54842b28085b39daa338f1e/users/5627ee24f25e40be96b9f575d368d8cf/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:19.553415 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4fde9-c849-47e4-80c9-7b56bc2e42d5 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:19.553720 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4fde9-c849-47e4-80c9-7b56bc2e42d5 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:19.554024 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4fde9-c849-47e4-80c9-7b56bc2e42d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:19.583428 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-a5f4fde9-c849-47e4-80c9-7b56bc2e42d5 None None] MFA Rules not processed for user `5627ee24f25e40be96b9f575d368d8cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:19.584666 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a5f4fde9-c849-47e4-80c9-7b56bc2e42d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:19.626758 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f796f83-07d0-471b-a8e6-36e3fcd1e7d1 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:19.629221 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-9f796f83-07d0-471b-a8e6-36e3fcd1e7d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:19.675660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1243/2504] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:19.680381 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4fde9-c849-47e4-80c9-7b56bc2e42d5 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:19.720115 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f796f83-07d0-471b-a8e6-36e3fcd1e7d1 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:19.727206 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07533841-f963-492b-a556-8db3573b1ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:19.728631 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07533841-f963-492b-a556-8db3573b1ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:19.729253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07533841-f963-492b-a556-8db3573b1ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:19.729751 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07533841-f963-492b-a556-8db3573b1ef7 admin admin] PATH_INFO: `/v3/users/80386d7afc994931af05fd49fa8043d6` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:19.731504 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07533841-f963-492b-a556-8db3573b1ef7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80386d7afc994931af05fd49fa8043d6)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:19.735063 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07533841-f963-492b-a556-8db3573b1ef7 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:19.762264 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:19.769287 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4fde9-c849-47e4-80c9-7b56bc2e42d5 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:19.776215 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92f066-e594-4752-bc6a-703287515b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:19.776972 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92f066-e594-4752-bc6a-703287515b21 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:19.777266 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92f066-e594-4752-bc6a-703287515b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:19.777541 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92f066-e594-4752-bc6a-703287515b21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:19.778271 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92f066-e594-4752-bc6a-703287515b21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:19.780997 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92f066-e594-4752-bc6a-703287515b21 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:19.800132 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-07533841-f963-492b-a556-8db3573b1ef7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80386d7afc994931af05fd49fa8043d6'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:19.808260 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-07533841-f963-492b-a556-8db3573b1ef7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80386d7afc994931af05fd49fa8043d6'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:19.822924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1244/2505] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:20:19 2022] POST /identity/v3/projects => generated 377 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:20:19.823131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-07533841-f963-492b-a556-8db3573b1ef7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80386d7afc994931af05fd49fa8043d6'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:19.823767 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-07533841-f963-492b-a556-8db3573b1ef7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80386d7afc994931af05fd49fa8043d6'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:19.823958 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92f066-e594-4752-bc6a-703287515b21 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:19.832776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-07533841-f963-492b-a556-8db3573b1ef7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80386d7afc994931af05fd49fa8043d6'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:19.834783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:19.841691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1262/2506] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:20:19 2022] DELETE /identity/v3/users/80386d7afc994931af05fd49fa8043d6 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:20:19.901984 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92f066-e594-4752-bc6a-703287515b21 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:19.975873 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b5d48-51fc-4cc5-948a-a9becbc36105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:19.976591 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b5d48-51fc-4cc5-948a-a9becbc36105 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:19.976891 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b5d48-51fc-4cc5-948a-a9becbc36105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:19.977157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b5d48-51fc-4cc5-948a-a9becbc36105 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:19.977911 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369b5d48-51fc-4cc5-948a-a9becbc36105 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:19.980020 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369b5d48-51fc-4cc5-948a-a9becbc36105 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:20.003684 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1245/2507] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:20:19 2022] POST /identity/v3/users => generated 295 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:20.009661 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07533841-f963-492b-a556-8db3573b1ef7 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:20.011168 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-07533841-f963-492b-a556-8db3573b1ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:20.027510 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07533841-f963-492b-a556-8db3573b1ef7 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:20.033680 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a651fd1-7b2c-4f60-809d-87bd9a198234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:20.035454 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a651fd1-7b2c-4f60-809d-87bd9a198234 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:20.035454 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a651fd1-7b2c-4f60-809d-87bd9a198234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:20.035454 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a651fd1-7b2c-4f60-809d-87bd9a198234 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:20.043960 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a651fd1-7b2c-4f60-809d-87bd9a198234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:20.043960 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a651fd1-7b2c-4f60-809d-87bd9a198234 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:20.044910 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b5d48-51fc-4cc5-948a-a9becbc36105 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:20.046449 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-369b5d48-51fc-4cc5-948a-a9becbc36105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:20.053468 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1263/2508] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:20:20.061201 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a651fd1-7b2c-4f60-809d-87bd9a198234 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:20.067489 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b5d48-51fc-4cc5-948a-a9becbc36105 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:20.072641 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f1dc3-2055-4cd2-84f7-840b2c28cfcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:20.074829 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f1dc3-2055-4cd2-84f7-840b2c28cfcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:20.075136 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f1dc3-2055-4cd2-84f7-840b2c28cfcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:20.075483 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f1dc3-2055-4cd2-84f7-840b2c28cfcc admin admin] PATH_INFO: `/v3/projects/34323d7e571b4f1db4d6e0c6cd1159cc` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:20.076369 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a651fd1-7b2c-4f60-809d-87bd9a198234 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:20.076783 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f1dc3-2055-4cd2-84f7-840b2c28cfcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=34323d7e571b4f1db4d6e0c6cd1159cc)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:20.079712 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f1dc3-2055-4cd2-84f7-840b2c28cfcc admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:20.082962 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddca571-39b0-4d5e-b096-c25e312395f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:20.084123 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddca571-39b0-4d5e-b096-c25e312395f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:20.084455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddca571-39b0-4d5e-b096-c25e312395f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:20.084803 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddca571-39b0-4d5e-b096-c25e312395f4 admin admin] PATH_INFO: `/v3/projects/3c7b7d47f1614c8a9cfa3e0b334c4cb5/users/ce5c6f6a5ff847aeb51cce3e61f42f16/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:20.102302 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddca571-39b0-4d5e-b096-c25e312395f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c7b7d47f1614c8a9cfa3e0b334c4cb5, user_id=ce5c6f6a5ff847aeb51cce3e61f42f16, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:20.105323 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddca571-39b0-4d5e-b096-c25e312395f4 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:20.120578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1264/2509] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:20 2022] PUT /identity/v3/projects/3c7b7d47f1614c8a9cfa3e0b334c4cb5/users/ce5c6f6a5ff847aeb51cce3e61f42f16/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:20.126419 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddca571-39b0-4d5e-b096-c25e312395f4 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:20.140146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d72f1dc3-2055-4cd2-84f7-840b2c28cfcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34323d7e571b4f1db4d6e0c6cd1159cc', 'request_id': 'req-d72f1dc3-2055-4cd2-84f7-840b2c28cfcc'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:20.150767 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d72f1dc3-2055-4cd2-84f7-840b2c28cfcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34323d7e571b4f1db4d6e0c6cd1159cc', 'request_id': 'req-d72f1dc3-2055-4cd2-84f7-840b2c28cfcc'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:20.163964 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d72f1dc3-2055-4cd2-84f7-840b2c28cfcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34323d7e571b4f1db4d6e0c6cd1159cc', 'request_id': 'req-d72f1dc3-2055-4cd2-84f7-840b2c28cfcc'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:20.174206 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:20.179036 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d72f1dc3-2055-4cd2-84f7-840b2c28cfcc admin admin] The token cache is being invalidate because project 34323d7e571b4f1db4d6e0c6cd1159cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:20:20.179372 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d72f1dc3-2055-4cd2-84f7-840b2c28cfcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:20.180667 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d72f1dc3-2055-4cd2-84f7-840b2c28cfcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:20.183001 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1246/2510] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:20:20 2022] DELETE /identity/v3/projects/34323d7e571b4f1db4d6e0c6cd1159cc => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:20:20.189002 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8e410-9ae1-4710-ae3e-e97ed40549db None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:20.189291 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8e410-9ae1-4710-ae3e-e97ed40549db None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:20.189575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8e410-9ae1-4710-ae3e-e97ed40549db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:20.197324 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddca571-39b0-4d5e-b096-c25e312395f4 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:20.213245 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f1c8e410-9ae1-4710-ae3e-e97ed40549db None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:20.214496 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f1c8e410-9ae1-4710-ae3e-e97ed40549db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:20.262683 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d15942-3109-4f24-bb47-e5e1f001fe25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:20.263361 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d15942-3109-4f24-bb47-e5e1f001fe25 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:20.263646 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d15942-3109-4f24-bb47-e5e1f001fe25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:20.263917 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d15942-3109-4f24-bb47-e5e1f001fe25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:20.266119 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d15942-3109-4f24-bb47-e5e1f001fe25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:20.267244 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d15942-3109-4f24-bb47-e5e1f001fe25 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:20.278215 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1265/2511] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:20:20.280014 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1247/2512] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:20:20.286887 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d15942-3109-4f24-bb47-e5e1f001fe25 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:20.287855 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee96b1-4037-42f5-a453-bac924b551de None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:20.288154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee96b1-4037-42f5-a453-bac924b551de None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:20.288467 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee96b1-4037-42f5-a453-bac924b551de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:20.288748 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-03d15942-3109-4f24-bb47-e5e1f001fe25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:20.302250 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d15942-3109-4f24-bb47-e5e1f001fe25 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:20.307951 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-80ee96b1-4037-42f5-a453-bac924b551de None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:20.309091 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-80ee96b1-4037-42f5-a453-bac924b551de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:20.311072 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7cff9-54e1-466f-83c1-9f64dcfc2fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:20.312722 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7cff9-54e1-466f-83c1-9f64dcfc2fb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:20.313124 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7cff9-54e1-466f-83c1-9f64dcfc2fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:20.313414 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7cff9-54e1-466f-83c1-9f64dcfc2fb1 admin admin] PATH_INFO: `/v3/projects/3c7b7d47f1614c8a9cfa3e0b334c4cb5/users/ce5c6f6a5ff847aeb51cce3e61f42f16/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:20.315515 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded7cff9-54e1-466f-83c1-9f64dcfc2fb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c7b7d47f1614c8a9cfa3e0b334c4cb5, user_id=ce5c6f6a5ff847aeb51cce3e61f42f16, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:20.318293 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded7cff9-54e1-466f-83c1-9f64dcfc2fb1 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:20.322411 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1248/2513] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:20:20.327008 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee96b1-4037-42f5-a453-bac924b551de None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:20.331634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1266/2514] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:20 2022] PUT /identity/v3/projects/3c7b7d47f1614c8a9cfa3e0b334c4cb5/users/ce5c6f6a5ff847aeb51cce3e61f42f16/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:20.337621 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee96b1-4037-42f5-a453-bac924b551de None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:20.341346 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ec0d3-a7a3-4691-850f-9219763f2838 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:20.341634 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ec0d3-a7a3-4691-850f-9219763f2838 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:20.341914 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ec0d3-a7a3-4691-850f-9219763f2838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:20.372552 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-2c7ec0d3-a7a3-4691-850f-9219763f2838 None None] MFA Rules not processed for user `ce5c6f6a5ff847aeb51cce3e61f42f16`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:20.373946 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-2c7ec0d3-a7a3-4691-850f-9219763f2838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:20.403356 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f24d32-6b1c-4dac-9f46-5fd6a84f2f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:20.404026 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f24d32-6b1c-4dac-9f46-5fd6a84f2f76 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:20.404305 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f24d32-6b1c-4dac-9f46-5fd6a84f2f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:20.404577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f24d32-6b1c-4dac-9f46-5fd6a84f2f76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:20.405394 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f24d32-6b1c-4dac-9f46-5fd6a84f2f76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:20:20.405697 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f24d32-6b1c-4dac-9f46-5fd6a84f2f76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:20.407347 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f24d32-6b1c-4dac-9f46-5fd6a84f2f76 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:20.424510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1249/2515] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:20:20.424777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:20.425959 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f24d32-6b1c-4dac-9f46-5fd6a84f2f76 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:20.436214 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f24d32-6b1c-4dac-9f46-5fd6a84f2f76 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:20.441614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61819edb-d3f5-41b6-881f-bf476366af30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:20.442325 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61819edb-d3f5-41b6-881f-bf476366af30 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:20.442634 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61819edb-d3f5-41b6-881f-bf476366af30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:20.442912 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61819edb-d3f5-41b6-881f-bf476366af30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:20.443631 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61819edb-d3f5-41b6-881f-bf476366af30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:20.445543 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61819edb-d3f5-41b6-881f-bf476366af30 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:20.456797 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1267/2516] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:20 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:20:20.469086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ec0d3-a7a3-4691-850f-9219763f2838 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:20.470117 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-2c7ec0d3-a7a3-4691-850f-9219763f2838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:20.478228 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1250/2517] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:20:20 2022] POST /identity/v3/projects => generated 367 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:20:20.482418 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61819edb-d3f5-41b6-881f-bf476366af30 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:20.530585 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ec0d3-a7a3-4691-850f-9219763f2838 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:20.557194 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61819edb-d3f5-41b6-881f-bf476366af30 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:20.564475 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002930a3-ee3d-4822-b357-e5b197db709d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:20.565080 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002930a3-ee3d-4822-b357-e5b197db709d admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:20.565373 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002930a3-ee3d-4822-b357-e5b197db709d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:20.565651 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002930a3-ee3d-4822-b357-e5b197db709d admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:20.566373 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002930a3-ee3d-4822-b357-e5b197db709d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:20.568317 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002930a3-ee3d-4822-b357-e5b197db709d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:20.579117 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:20.587267 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5688930-bbda-46e3-9e55-6e07ada72486 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:20.588329 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5688930-bbda-46e3-9e55-6e07ada72486 service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:20.588631 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5688930-bbda-46e3-9e55-6e07ada72486 service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:20.588910 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5688930-bbda-46e3-9e55-6e07ada72486 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:20.591371 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1251/2518] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:20:20 2022] POST /identity/v3/users => generated 284 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:20:20.595647 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002930a3-ee3d-4822-b357-e5b197db709d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:20.606505 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002930a3-ee3d-4822-b357-e5b197db709d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:20.612767 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b62cc-b4dc-4e5c-8262-8d5461debcd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:20.613447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b62cc-b4dc-4e5c-8262-8d5461debcd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:20.613735 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b62cc-b4dc-4e5c-8262-8d5461debcd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:20.614063 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b62cc-b4dc-4e5c-8262-8d5461debcd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:20.614938 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b62cc-b4dc-4e5c-8262-8d5461debcd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:20.617028 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b62cc-b4dc-4e5c-8262-8d5461debcd7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:20.628723 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1252/2519] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:20:20.632726 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b62cc-b4dc-4e5c-8262-8d5461debcd7 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:20.643833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b62cc-b4dc-4e5c-8262-8d5461debcd7 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:20.649038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df00e12-eec2-4469-a1d0-e777b9fa724f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:20.649671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df00e12-eec2-4469-a1d0-e777b9fa724f admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:20.649993 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df00e12-eec2-4469-a1d0-e777b9fa724f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:20.650460 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df00e12-eec2-4469-a1d0-e777b9fa724f admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:20.651140 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df00e12-eec2-4469-a1d0-e777b9fa724f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:20.652767 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df00e12-eec2-4469-a1d0-e777b9fa724f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:20.664912 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1253/2520] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:20:20.669203 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5688930-bbda-46e3-9e55-6e07ada72486 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:20.670349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df00e12-eec2-4469-a1d0-e777b9fa724f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:20.671077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5688930-bbda-46e3-9e55-6e07ada72486 service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:20.680554 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df00e12-eec2-4469-a1d0-e777b9fa724f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:20.682196 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1268/2521] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:20 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:20:20.688311 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18baf345-4330-4ec2-a416-7181655b7ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:20.689235 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18baf345-4330-4ec2-a416-7181655b7ac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:20.689569 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18baf345-4330-4ec2-a416-7181655b7ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:20.689854 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18baf345-4330-4ec2-a416-7181655b7ac7 admin admin] PATH_INFO: `/v3/projects/85aac9178f464a7483b313ee1343ef25/users/e2481546c9d84c1d810860978faa6b28/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:20.707477 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18baf345-4330-4ec2-a416-7181655b7ac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85aac9178f464a7483b313ee1343ef25, user_id=e2481546c9d84c1d810860978faa6b28, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:20.710190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18baf345-4330-4ec2-a416-7181655b7ac7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:20.726509 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1254/2522] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:20 2022] PUT /identity/v3/projects/85aac9178f464a7483b313ee1343ef25/users/e2481546c9d84c1d810860978faa6b28/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:20:20.733340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5688930-bbda-46e3-9e55-6e07ada72486 service cinder] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:20.735536 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-f5688930-bbda-46e3-9e55-6e07ada72486 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:20.737373 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a28b8-ca33-49cc-9e1b-d3ed3d114ad3 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:20.737672 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a28b8-ca33-49cc-9e1b-d3ed3d114ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:20.737977 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a28b8-ca33-49cc-9e1b-d3ed3d114ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:20.750334 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5688930-bbda-46e3-9e55-6e07ada72486 service cinder] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:20.770316 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-045a28b8-ca33-49cc-9e1b-d3ed3d114ad3 None None] MFA Rules not processed for user `e2481546c9d84c1d810860978faa6b28`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:20.771604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-045a28b8-ca33-49cc-9e1b-d3ed3d114ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:20.819002 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:20.839968 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0f8eb-1fb4-4b97-80ef-7797c304ca5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:20.841604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0f8eb-1fb4-4b97-80ef-7797c304ca5c service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:20.841604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0f8eb-1fb4-4b97-80ef-7797c304ca5c service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:20.841604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0f8eb-1fb4-4b97-80ef-7797c304ca5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:20.844299 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-8cb0f8eb-1fb4-4b97-80ef-7797c304ca5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:20.860646 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1255/2523] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:20 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:20:20.872724 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a28b8-ca33-49cc-9e1b-d3ed3d114ad3 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:20.885034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a28b8-ca33-49cc-9e1b-d3ed3d114ad3 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:20.889257 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be683ecb-2861-4f6f-8d9e-0cb1f56927ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:20.890962 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be683ecb-2861-4f6f-8d9e-0cb1f56927ad service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:20.891610 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be683ecb-2861-4f6f-8d9e-0cb1f56927ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:20.892173 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be683ecb-2861-4f6f-8d9e-0cb1f56927ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:20.931876 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be683ecb-2861-4f6f-8d9e-0cb1f56927ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:20.933754 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be683ecb-2861-4f6f-8d9e-0cb1f56927ad service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:20.942858 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb0f8eb-1fb4-4b97-80ef-7797c304ca5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:20.944193 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb0f8eb-1fb4-4b97-80ef-7797c304ca5c service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:20.948094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1256/2524] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:20 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:20:20.955729 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1269/2525] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:20:24.060787 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be683ecb-2861-4f6f-8d9e-0cb1f56927ad service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:24.136191 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be683ecb-2861-4f6f-8d9e-0cb1f56927ad service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:24.141229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfee143-87ee-40d1-b10b-a71fa6c31c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:24.142544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfee143-87ee-40d1-b10b-a71fa6c31c2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:24.142672 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfee143-87ee-40d1-b10b-a71fa6c31c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:24.143110 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfee143-87ee-40d1-b10b-a71fa6c31c2c admin admin] PATH_INFO: `/v3/users/f25373331d774b6db279285644773e7b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:24.145135 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfee143-87ee-40d1-b10b-a71fa6c31c2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f25373331d774b6db279285644773e7b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:24.146823 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfee143-87ee-40d1-b10b-a71fa6c31c2c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:24.191714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:24.215008 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2dfee143-87ee-40d1-b10b-a71fa6c31c2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f25373331d774b6db279285644773e7b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:24.231182 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2dfee143-87ee-40d1-b10b-a71fa6c31c2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f25373331d774b6db279285644773e7b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:24.242081 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2dfee143-87ee-40d1-b10b-a71fa6c31c2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f25373331d774b6db279285644773e7b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:24.250934 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2dfee143-87ee-40d1-b10b-a71fa6c31c2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f25373331d774b6db279285644773e7b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:24.251730 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2dfee143-87ee-40d1-b10b-a71fa6c31c2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f25373331d774b6db279285644773e7b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:24.257994 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1257/2526] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:20:24 2022] DELETE /identity/v3/users/f25373331d774b6db279285644773e7b => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:24.262801 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f56fac-bd51-45f5-ad3b-ec74f5c68838 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:24.263319 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f56fac-bd51-45f5-ad3b-ec74f5c68838 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:24.263435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f56fac-bd51-45f5-ad3b-ec74f5c68838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:24.283530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-a1f56fac-bd51-45f5-ad3b-ec74f5c68838 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:24.284782 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-a1f56fac-bd51-45f5-ad3b-ec74f5c68838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:24.356109 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1270/2527] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:24.366475 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfee143-87ee-40d1-b10b-a71fa6c31c2c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:24.367966 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2dfee143-87ee-40d1-b10b-a71fa6c31c2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:24.440452 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfee143-87ee-40d1-b10b-a71fa6c31c2c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:24.446894 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98713327-818b-475d-8228-16cc8ae51d46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:24.448001 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98713327-818b-475d-8228-16cc8ae51d46 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:24.448512 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98713327-818b-475d-8228-16cc8ae51d46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:24.449008 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98713327-818b-475d-8228-16cc8ae51d46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:24.462732 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98713327-818b-475d-8228-16cc8ae51d46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:24.464368 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98713327-818b-475d-8228-16cc8ae51d46 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:24.476577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1258/2528] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:20:24.774966 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f56fac-bd51-45f5-ad3b-ec74f5c68838 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:24.776450 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-a1f56fac-bd51-45f5-ad3b-ec74f5c68838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:24.796843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f56fac-bd51-45f5-ad3b-ec74f5c68838 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:24.803733 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab930b-ca3b-45df-b16e-6e184ca5b961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:24.804620 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab930b-ca3b-45df-b16e-6e184ca5b961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:24.804907 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab930b-ca3b-45df-b16e-6e184ca5b961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:24.805290 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab930b-ca3b-45df-b16e-6e184ca5b961 admin admin] PATH_INFO: `/v3/projects/32ce91baa96b41128461ad8c72e68c27` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:24.806604 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab930b-ca3b-45df-b16e-6e184ca5b961 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32ce91baa96b41128461ad8c72e68c27)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:24.809220 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab930b-ca3b-45df-b16e-6e184ca5b961 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:24.867415 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-02ab930b-ca3b-45df-b16e-6e184ca5b961 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32ce91baa96b41128461ad8c72e68c27', 'request_id': 'req-02ab930b-ca3b-45df-b16e-6e184ca5b961'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:24.874330 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-02ab930b-ca3b-45df-b16e-6e184ca5b961 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32ce91baa96b41128461ad8c72e68c27', 'request_id': 'req-02ab930b-ca3b-45df-b16e-6e184ca5b961'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:24.883261 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-02ab930b-ca3b-45df-b16e-6e184ca5b961 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32ce91baa96b41128461ad8c72e68c27', 'request_id': 'req-02ab930b-ca3b-45df-b16e-6e184ca5b961'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:24.896246 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-02ab930b-ca3b-45df-b16e-6e184ca5b961 admin admin] The token cache is being invalidate because project 32ce91baa96b41128461ad8c72e68c27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:20:24.896532 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-02ab930b-ca3b-45df-b16e-6e184ca5b961 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:24.897245 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-02ab930b-ca3b-45df-b16e-6e184ca5b961 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:24.898914 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1271/2529] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:20:24 2022] DELETE /identity/v3/projects/32ce91baa96b41128461ad8c72e68c27 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:24.902901 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98713327-818b-475d-8228-16cc8ae51d46 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:24.913268 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98713327-818b-475d-8228-16cc8ae51d46 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:24.974469 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100ba5a-c1f9-4e10-9dd2-60130529256e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:24.975342 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100ba5a-c1f9-4e10-9dd2-60130529256e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:24.975618 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100ba5a-c1f9-4e10-9dd2-60130529256e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:24.975897 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100ba5a-c1f9-4e10-9dd2-60130529256e admin admin] PATH_INFO: `/v3/users/5627ee24f25e40be96b9f575d368d8cf` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:24.977084 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6100ba5a-c1f9-4e10-9dd2-60130529256e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5627ee24f25e40be96b9f575d368d8cf)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:24.979036 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6100ba5a-c1f9-4e10-9dd2-60130529256e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:25.036242 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6100ba5a-c1f9-4e10-9dd2-60130529256e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5627ee24f25e40be96b9f575d368d8cf'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:25.038425 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:25.052665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6100ba5a-c1f9-4e10-9dd2-60130529256e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5627ee24f25e40be96b9f575d368d8cf'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:25.059906 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6100ba5a-c1f9-4e10-9dd2-60130529256e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5627ee24f25e40be96b9f575d368d8cf'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:25.068917 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6100ba5a-c1f9-4e10-9dd2-60130529256e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5627ee24f25e40be96b9f575d368d8cf'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:25.069643 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6100ba5a-c1f9-4e10-9dd2-60130529256e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5627ee24f25e40be96b9f575d368d8cf'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:25.075759 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1259/2530] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:20:24 2022] DELETE /identity/v3/users/5627ee24f25e40be96b9f575d368d8cf => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:25.398501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab930b-ca3b-45df-b16e-6e184ca5b961 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:25.400548 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-02ab930b-ca3b-45df-b16e-6e184ca5b961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:25.415779 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab930b-ca3b-45df-b16e-6e184ca5b961 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:25.453177 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:25.499927 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9968b2-c489-42bb-a6c9-b9ee0010ab23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:25.500930 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9968b2-c489-42bb-a6c9-b9ee0010ab23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:25.501271 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9968b2-c489-42bb-a6c9-b9ee0010ab23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:25.501612 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9968b2-c489-42bb-a6c9-b9ee0010ab23 admin admin] PATH_INFO: `/v3/projects/4ddec6e8e54842b28085b39daa338f1e` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:25.502945 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9968b2-c489-42bb-a6c9-b9ee0010ab23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ddec6e8e54842b28085b39daa338f1e)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:25.504959 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9968b2-c489-42bb-a6c9-b9ee0010ab23 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:25.565562 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-cf9968b2-c489-42bb-a6c9-b9ee0010ab23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ddec6e8e54842b28085b39daa338f1e', 'request_id': 'req-cf9968b2-c489-42bb-a6c9-b9ee0010ab23'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:25.571936 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-cf9968b2-c489-42bb-a6c9-b9ee0010ab23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ddec6e8e54842b28085b39daa338f1e', 'request_id': 'req-cf9968b2-c489-42bb-a6c9-b9ee0010ab23'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:25.580155 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-cf9968b2-c489-42bb-a6c9-b9ee0010ab23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ddec6e8e54842b28085b39daa338f1e', 'request_id': 'req-cf9968b2-c489-42bb-a6c9-b9ee0010ab23'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:25.593526 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-cf9968b2-c489-42bb-a6c9-b9ee0010ab23 admin admin] The token cache is being invalidate because project 4ddec6e8e54842b28085b39daa338f1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:20:25.593889 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-cf9968b2-c489-42bb-a6c9-b9ee0010ab23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:25.594679 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-cf9968b2-c489-42bb-a6c9-b9ee0010ab23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:25.596388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1272/2531] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:20:25 2022] DELETE /identity/v3/projects/4ddec6e8e54842b28085b39daa338f1e => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:20:25.601037 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100ba5a-c1f9-4e10-9dd2-60130529256e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:25.603256 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6100ba5a-c1f9-4e10-9dd2-60130529256e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:25.618621 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100ba5a-c1f9-4e10-9dd2-60130529256e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:25.688994 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac7e82-ee31-4b05-9ee4-1d9aded7fbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:25.689800 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac7e82-ee31-4b05-9ee4-1d9aded7fbe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:25.690199 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac7e82-ee31-4b05-9ee4-1d9aded7fbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:25.690425 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac7e82-ee31-4b05-9ee4-1d9aded7fbe0 admin admin] PATH_INFO: `/v3/users/ce5c6f6a5ff847aeb51cce3e61f42f16` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:25.691657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bac7e82-ee31-4b05-9ee4-1d9aded7fbe0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce5c6f6a5ff847aeb51cce3e61f42f16)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:25.693619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bac7e82-ee31-4b05-9ee4-1d9aded7fbe0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:25.752962 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9bac7e82-ee31-4b05-9ee4-1d9aded7fbe0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce5c6f6a5ff847aeb51cce3e61f42f16'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:25.770122 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9bac7e82-ee31-4b05-9ee4-1d9aded7fbe0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce5c6f6a5ff847aeb51cce3e61f42f16'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:25.779324 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9bac7e82-ee31-4b05-9ee4-1d9aded7fbe0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce5c6f6a5ff847aeb51cce3e61f42f16'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:25.791326 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9bac7e82-ee31-4b05-9ee4-1d9aded7fbe0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce5c6f6a5ff847aeb51cce3e61f42f16'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:25.792921 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9bac7e82-ee31-4b05-9ee4-1d9aded7fbe0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce5c6f6a5ff847aeb51cce3e61f42f16'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:25.802849 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1260/2532] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:20:25 2022] DELETE /identity/v3/users/ce5c6f6a5ff847aeb51cce3e61f42f16 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:20:26.118367 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9968b2-c489-42bb-a6c9-b9ee0010ab23 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:26.119824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-cf9968b2-c489-42bb-a6c9-b9ee0010ab23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:26.135186 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9968b2-c489-42bb-a6c9-b9ee0010ab23 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:26.214342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfbb974-853e-4060-a0ad-af1c53d5901b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:26.215149 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbb974-853e-4060-a0ad-af1c53d5901b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:26.215438 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbb974-853e-4060-a0ad-af1c53d5901b admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:26.215724 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbb974-853e-4060-a0ad-af1c53d5901b admin admin] PATH_INFO: `/v3/projects/3c7b7d47f1614c8a9cfa3e0b334c4cb5` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:26.216989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfbb974-853e-4060-a0ad-af1c53d5901b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c7b7d47f1614c8a9cfa3e0b334c4cb5)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:26.219058 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfbb974-853e-4060-a0ad-af1c53d5901b admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:26.295379 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-cdfbb974-853e-4060-a0ad-af1c53d5901b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c7b7d47f1614c8a9cfa3e0b334c4cb5', 'request_id': 'req-cdfbb974-853e-4060-a0ad-af1c53d5901b'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:26.299616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:26.302090 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-cdfbb974-853e-4060-a0ad-af1c53d5901b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c7b7d47f1614c8a9cfa3e0b334c4cb5', 'request_id': 'req-cdfbb974-853e-4060-a0ad-af1c53d5901b'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:26.311760 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-cdfbb974-853e-4060-a0ad-af1c53d5901b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c7b7d47f1614c8a9cfa3e0b334c4cb5', 'request_id': 'req-cdfbb974-853e-4060-a0ad-af1c53d5901b'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:26.324875 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-cdfbb974-853e-4060-a0ad-af1c53d5901b admin admin] The token cache is being invalidate because project 3c7b7d47f1614c8a9cfa3e0b334c4cb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:20:26.325612 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-cdfbb974-853e-4060-a0ad-af1c53d5901b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:26.326243 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-cdfbb974-853e-4060-a0ad-af1c53d5901b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:26.329122 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1273/2533] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:20:26 2022] DELETE /identity/v3/projects/3c7b7d47f1614c8a9cfa3e0b334c4cb5 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:26.335528 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb764a-edc5-448e-96bc-c2016c1db8db None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:26.336085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb764a-edc5-448e-96bc-c2016c1db8db None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:26.336085 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb764a-edc5-448e-96bc-c2016c1db8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:26.364966 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-17fb764a-edc5-448e-96bc-c2016c1db8db None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:26.366226 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-17fb764a-edc5-448e-96bc-c2016c1db8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:26.418862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:26.442208 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1261/2534] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:26.453316 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2f91a-884d-460c-a0de-068d1c18522e None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:26.454174 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2f91a-884d-460c-a0de-068d1c18522e None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:26.454174 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2f91a-884d-460c-a0de-068d1c18522e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:26.475806 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-46d2f91a-884d-460c-a0de-068d1c18522e None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:26.477067 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-46d2f91a-884d-460c-a0de-068d1c18522e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:26.486286 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1274/2535] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:26.490761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb764a-edc5-448e-96bc-c2016c1db8db None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:26.501270 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb764a-edc5-448e-96bc-c2016c1db8db None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:26.506222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6321702-dbc4-4c2f-bf0d-61328fc3ad3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:26.506886 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6321702-dbc4-4c2f-bf0d-61328fc3ad3f admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:26.507204 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6321702-dbc4-4c2f-bf0d-61328fc3ad3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:26.507487 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6321702-dbc4-4c2f-bf0d-61328fc3ad3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:26.508219 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6321702-dbc4-4c2f-bf0d-61328fc3ad3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:20:26.508531 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6321702-dbc4-4c2f-bf0d-61328fc3ad3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:26.510449 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6321702-dbc4-4c2f-bf0d-61328fc3ad3f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:26.522723 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1262/2536] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:20:26.527325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2f91a-884d-460c-a0de-068d1c18522e None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:26.539619 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2f91a-884d-460c-a0de-068d1c18522e None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:26.544917 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8062f-b93d-4764-b07d-ce99df67028e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:26.546310 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8062f-b93d-4764-b07d-ce99df67028e admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:26.547128 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8062f-b93d-4764-b07d-ce99df67028e admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:26.547408 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8062f-b93d-4764-b07d-ce99df67028e admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:26.548134 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e8062f-b93d-4764-b07d-ce99df67028e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:26.550108 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e8062f-b93d-4764-b07d-ce99df67028e admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:26.578536 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1275/2537] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:20:26 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:26.583623 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6321702-dbc4-4c2f-bf0d-61328fc3ad3f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:26.596303 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6321702-dbc4-4c2f-bf0d-61328fc3ad3f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:26.676945 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd89a9-ce3b-4f87-bde9-de6e184f0bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:26.677693 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd89a9-ce3b-4f87-bde9-de6e184f0bc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:26.678040 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd89a9-ce3b-4f87-bde9-de6e184f0bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:26.678349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd89a9-ce3b-4f87-bde9-de6e184f0bc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:26.679072 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fd89a9-ce3b-4f87-bde9-de6e184f0bc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:26.681191 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fd89a9-ce3b-4f87-bde9-de6e184f0bc5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:26.705698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1263/2538] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:20:26 2022] POST /identity/v3/users => generated 286 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:26.710978 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8062f-b93d-4764-b07d-ce99df67028e admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:26.722609 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8062f-b93d-4764-b07d-ce99df67028e admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:26.727619 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a850383-9972-46bd-b9f6-832aca71d645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:26.728290 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a850383-9972-46bd-b9f6-832aca71d645 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:26.728581 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a850383-9972-46bd-b9f6-832aca71d645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:26.728860 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a850383-9972-46bd-b9f6-832aca71d645 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:26.729713 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a850383-9972-46bd-b9f6-832aca71d645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:26.731477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a850383-9972-46bd-b9f6-832aca71d645 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:26.744620 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1276/2539] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:26.750720 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd89a9-ce3b-4f87-bde9-de6e184f0bc5 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:26.763475 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd89a9-ce3b-4f87-bde9-de6e184f0bc5 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:26.769374 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e83256-9a2b-4afa-9637-a8dc83a53cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:26.770151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e83256-9a2b-4afa-9637-a8dc83a53cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:26.770475 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e83256-9a2b-4afa-9637-a8dc83a53cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:26.770903 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e83256-9a2b-4afa-9637-a8dc83a53cf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:26.771681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e83256-9a2b-4afa-9637-a8dc83a53cf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:26.773462 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e83256-9a2b-4afa-9637-a8dc83a53cf5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:26.787693 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1264/2540] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:26.792642 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a850383-9972-46bd-b9f6-832aca71d645 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:26.805829 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a850383-9972-46bd-b9f6-832aca71d645 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:26.811462 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f796d-9ff0-4f43-9f7c-ad424b94814a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:26.812357 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f796d-9ff0-4f43-9f7c-ad424b94814a admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:26.812642 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f796d-9ff0-4f43-9f7c-ad424b94814a admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:26.812922 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f796d-9ff0-4f43-9f7c-ad424b94814a admin admin] PATH_INFO: `/v3/projects/765cab7190f642e891d2e7236edc3a8a/users/a76571e533064b1ba007b5d1a025c3cf/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:26.829677 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8f796d-9ff0-4f43-9f7c-ad424b94814a admin admin] RBAC: Authorizing `identity:create_grant(project_id=765cab7190f642e891d2e7236edc3a8a, user_id=a76571e533064b1ba007b5d1a025c3cf, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:26.832638 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8f796d-9ff0-4f43-9f7c-ad424b94814a admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:26.847976 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1277/2541] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:26 2022] PUT /identity/v3/projects/765cab7190f642e891d2e7236edc3a8a/users/a76571e533064b1ba007b5d1a025c3cf/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:26.862090 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54875b2e-6d1b-481e-8e45-ca36a1d3eb62 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:26.862383 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54875b2e-6d1b-481e-8e45-ca36a1d3eb62 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:26.862662 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54875b2e-6d1b-481e-8e45-ca36a1d3eb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:26.893330 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-54875b2e-6d1b-481e-8e45-ca36a1d3eb62 None None] MFA Rules not processed for user `a76571e533064b1ba007b5d1a025c3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:26.894654 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-54875b2e-6d1b-481e-8e45-ca36a1d3eb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:26.966034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1265/2542] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:20:26.982064 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f796d-9ff0-4f43-9f7c-ad424b94814a admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:26.985095 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-8b8f796d-9ff0-4f43-9f7c-ad424b94814a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:27.059804 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f796d-9ff0-4f43-9f7c-ad424b94814a admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:27.064464 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b0be6-6c28-48b0-b168-ea88ec05d0cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:27.065114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b0be6-6c28-48b0-b168-ea88ec05d0cb service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:27.065419 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b0be6-6c28-48b0-b168-ea88ec05d0cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:27.065739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b0be6-6c28-48b0-b168-ea88ec05d0cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:27.076172 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b0be6-6c28-48b0-b168-ea88ec05d0cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:27.078399 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b0be6-6c28-48b0-b168-ea88ec05d0cb service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:27.088940 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1278/2543] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:20:28.481990 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9281635-051d-47d9-b224-74c05aab3353 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:28.482501 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9281635-051d-47d9-b224-74c05aab3353 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:28.482620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9281635-051d-47d9-b224-74c05aab3353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:28.502695 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e9281635-051d-47d9-b224-74c05aab3353 None None] MFA Rules not processed for user `a76571e533064b1ba007b5d1a025c3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:28.503923 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e9281635-051d-47d9-b224-74c05aab3353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:28.511188 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1266/2544] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:29.636388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b0be6-6c28-48b0-b168-ea88ec05d0cb service cinder] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:29.637863 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-1b8b0be6-6c28-48b0-b168-ea88ec05d0cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:29.711309 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b0be6-6c28-48b0-b168-ea88ec05d0cb service cinder] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:29.715517 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ce28a-2dbe-4cdf-97bd-f1edd778569c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:29.716439 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ce28a-2dbe-4cdf-97bd-f1edd778569c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:29.716747 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ce28a-2dbe-4cdf-97bd-f1edd778569c admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:29.717009 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ce28a-2dbe-4cdf-97bd-f1edd778569c admin admin] PATH_INFO: `/v3/users/e2481546c9d84c1d810860978faa6b28` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:29.718330 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ce28a-2dbe-4cdf-97bd-f1edd778569c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2481546c9d84c1d810860978faa6b28)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:29.720310 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ce28a-2dbe-4cdf-97bd-f1edd778569c admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:29.747502 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:29.788997 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-a00ce28a-2dbe-4cdf-97bd-f1edd778569c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2481546c9d84c1d810860978faa6b28'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:29.798136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-a00ce28a-2dbe-4cdf-97bd-f1edd778569c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2481546c9d84c1d810860978faa6b28'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:29.810955 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-a00ce28a-2dbe-4cdf-97bd-f1edd778569c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2481546c9d84c1d810860978faa6b28'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:29.812132 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-a00ce28a-2dbe-4cdf-97bd-f1edd778569c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2481546c9d84c1d810860978faa6b28'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:29.820695 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-a00ce28a-2dbe-4cdf-97bd-f1edd778569c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2481546c9d84c1d810860978faa6b28'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:29.830158 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1279/2545] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:20:29 2022] DELETE /identity/v3/users/e2481546c9d84c1d810860978faa6b28 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:20:29.834986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d641000e-2930-49a5-8893-dd819e82d299 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:29.836008 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d641000e-2930-49a5-8893-dd819e82d299 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:29.836008 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d641000e-2930-49a5-8893-dd819e82d299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:29.856017 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d641000e-2930-49a5-8893-dd819e82d299 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:29.857593 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d641000e-2930-49a5-8893-dd819e82d299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:29.896248 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:29.932935 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1267/2546] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:29.946581 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ce28a-2dbe-4cdf-97bd-f1edd778569c admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:29.948476 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-a00ce28a-2dbe-4cdf-97bd-f1edd778569c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:30.028744 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ce28a-2dbe-4cdf-97bd-f1edd778569c admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:30.034616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46173ee1-45bf-46d9-8a50-29051d1a0766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:30.035483 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46173ee1-45bf-46d9-8a50-29051d1a0766 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:30.036109 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46173ee1-45bf-46d9-8a50-29051d1a0766 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:30.036615 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46173ee1-45bf-46d9-8a50-29051d1a0766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:30.050667 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46173ee1-45bf-46d9-8a50-29051d1a0766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:30.052312 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46173ee1-45bf-46d9-8a50-29051d1a0766 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:30.067204 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1280/2547] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:20:30.670520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d641000e-2930-49a5-8893-dd819e82d299 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:30.672654 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d641000e-2930-49a5-8893-dd819e82d299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:30.685617 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d641000e-2930-49a5-8893-dd819e82d299 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:30.692538 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d659e7cb-606b-42d5-96b2-5b1a6465986e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:30.694660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659e7cb-606b-42d5-96b2-5b1a6465986e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:30.694949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659e7cb-606b-42d5-96b2-5b1a6465986e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:30.695237 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659e7cb-606b-42d5-96b2-5b1a6465986e admin admin] PATH_INFO: `/v3/projects/85aac9178f464a7483b313ee1343ef25` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:30.696728 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d659e7cb-606b-42d5-96b2-5b1a6465986e admin admin] RBAC: Authorizing `identity:delete_project(project_id=85aac9178f464a7483b313ee1343ef25)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:30.699105 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d659e7cb-606b-42d5-96b2-5b1a6465986e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:30.800438 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d659e7cb-606b-42d5-96b2-5b1a6465986e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85aac9178f464a7483b313ee1343ef25', 'request_id': 'req-d659e7cb-606b-42d5-96b2-5b1a6465986e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:30.810348 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d659e7cb-606b-42d5-96b2-5b1a6465986e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85aac9178f464a7483b313ee1343ef25', 'request_id': 'req-d659e7cb-606b-42d5-96b2-5b1a6465986e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:30.821831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d659e7cb-606b-42d5-96b2-5b1a6465986e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85aac9178f464a7483b313ee1343ef25', 'request_id': 'req-d659e7cb-606b-42d5-96b2-5b1a6465986e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:30.838668 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d659e7cb-606b-42d5-96b2-5b1a6465986e admin admin] The token cache is being invalidate because project 85aac9178f464a7483b313ee1343ef25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:20:30.839772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d659e7cb-606b-42d5-96b2-5b1a6465986e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:30.840112 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d659e7cb-606b-42d5-96b2-5b1a6465986e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:30.842081 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1268/2548] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:20:30 2022] DELETE /identity/v3/projects/85aac9178f464a7483b313ee1343ef25 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:30.847618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139760f1-5be9-4c30-b474-54c1f600bea2 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:30.848170 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139760f1-5be9-4c30-b474-54c1f600bea2 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:30.848269 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139760f1-5be9-4c30-b474-54c1f600bea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:30.867523 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-139760f1-5be9-4c30-b474-54c1f600bea2 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:30.868767 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-139760f1-5be9-4c30-b474-54c1f600bea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:30.940462 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1281/2549] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:30.948631 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81d73f-a497-4c26-9fd8-2e7231d98936 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:30.949000 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81d73f-a497-4c26-9fd8-2e7231d98936 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:30.949355 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81d73f-a497-4c26-9fd8-2e7231d98936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:30.969028 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8f81d73f-a497-4c26-9fd8-2e7231d98936 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:30.970694 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8f81d73f-a497-4c26-9fd8-2e7231d98936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:30.987424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1269/2550] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:20:30.992251 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139760f1-5be9-4c30-b474-54c1f600bea2 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:31.011608 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139760f1-5be9-4c30-b474-54c1f600bea2 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:31.017861 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5996f222-dbfb-477b-b85b-98044b0659ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:31.018575 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996f222-dbfb-477b-b85b-98044b0659ee admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:31.019110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996f222-dbfb-477b-b85b-98044b0659ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:31.019396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996f222-dbfb-477b-b85b-98044b0659ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:31.023496 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5996f222-dbfb-477b-b85b-98044b0659ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:20:31.023984 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5996f222-dbfb-477b-b85b-98044b0659ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:31.025984 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5996f222-dbfb-477b-b85b-98044b0659ee admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:31.051177 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1282/2551] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:20:31.055207 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f81d73f-a497-4c26-9fd8-2e7231d98936 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:31.067108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f81d73f-a497-4c26-9fd8-2e7231d98936 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:31.072344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1527d13-3598-40c7-b48d-c99e4010a299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:31.073388 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1527d13-3598-40c7-b48d-c99e4010a299 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:31.073744 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1527d13-3598-40c7-b48d-c99e4010a299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:31.074060 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1527d13-3598-40c7-b48d-c99e4010a299 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:31.074768 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1527d13-3598-40c7-b48d-c99e4010a299 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:31.077138 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1527d13-3598-40c7-b48d-c99e4010a299 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:31.113376 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1270/2552] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:20:31 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:31.115455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5996f222-dbfb-477b-b85b-98044b0659ee admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:31.126856 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5996f222-dbfb-477b-b85b-98044b0659ee admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:31.168629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:31.213663 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f94a1-e032-433d-86ac-20313a0afa3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:31.214443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f94a1-e032-433d-86ac-20313a0afa3c admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:31.214780 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f94a1-e032-433d-86ac-20313a0afa3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:31.215078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f94a1-e032-433d-86ac-20313a0afa3c admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:31.215752 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9f94a1-e032-433d-86ac-20313a0afa3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:31.218086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9f94a1-e032-433d-86ac-20313a0afa3c admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:31.260397 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1527d13-3598-40c7-b48d-c99e4010a299 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:31.261067 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1283/2553] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:20:31 2022] POST /identity/v3/users => generated 283 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:31.275552 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1527d13-3598-40c7-b48d-c99e4010a299 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:31.282098 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c83346-1919-4cb2-865c-1602d5c9c9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:31.283057 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c83346-1919-4cb2-865c-1602d5c9c9b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:31.283503 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c83346-1919-4cb2-865c-1602d5c9c9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:31.283963 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c83346-1919-4cb2-865c-1602d5c9c9b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:31.285069 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c83346-1919-4cb2-865c-1602d5c9c9b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:31.287030 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c83346-1919-4cb2-865c-1602d5c9c9b8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:31.300183 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1271/2554] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:31.304689 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f94a1-e032-433d-86ac-20313a0afa3c admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:31.319331 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f94a1-e032-433d-86ac-20313a0afa3c admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:31.326855 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7330fa-d769-4305-af0e-e3ada4eb6ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:31.327949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7330fa-d769-4305-af0e-e3ada4eb6ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:31.328456 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7330fa-d769-4305-af0e-e3ada4eb6ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:31.328950 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7330fa-d769-4305-af0e-e3ada4eb6ae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:31.330891 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7330fa-d769-4305-af0e-e3ada4eb6ae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:31.332840 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7330fa-d769-4305-af0e-e3ada4eb6ae1 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:31.354010 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1284/2555] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:31.357816 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c83346-1919-4cb2-865c-1602d5c9c9b8 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:31.371821 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c83346-1919-4cb2-865c-1602d5c9c9b8 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:31.377199 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d580cc-74d2-48fb-a991-885d22c57425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:31.378252 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d580cc-74d2-48fb-a991-885d22c57425 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:31.378686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d580cc-74d2-48fb-a991-885d22c57425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:31.378948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d580cc-74d2-48fb-a991-885d22c57425 admin admin] PATH_INFO: `/v3/projects/40cbc41c87954c02a7f3a27e60dafc65/users/83d6910c1cbd488a9c716176998ed5c5/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:31.397040 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d580cc-74d2-48fb-a991-885d22c57425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40cbc41c87954c02a7f3a27e60dafc65, user_id=83d6910c1cbd488a9c716176998ed5c5, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:31.399839 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d580cc-74d2-48fb-a991-885d22c57425 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:31.425149 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1272/2556] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:31 2022] PUT /identity/v3/projects/40cbc41c87954c02a7f3a27e60dafc65/users/83d6910c1cbd488a9c716176998ed5c5/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:31.436292 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babd0bf-f65b-4acf-95c8-619ec44d8e12 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:31.436567 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babd0bf-f65b-4acf-95c8-619ec44d8e12 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:31.436934 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babd0bf-f65b-4acf-95c8-619ec44d8e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:31.466777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-3babd0bf-f65b-4acf-95c8-619ec44d8e12 None None] MFA Rules not processed for user `83d6910c1cbd488a9c716176998ed5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:31.468698 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-3babd0bf-f65b-4acf-95c8-619ec44d8e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:31.549268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1285/2557] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:31 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:20:31.550433 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d580cc-74d2-48fb-a991-885d22c57425 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:31.637524 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:31.644484 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d580cc-74d2-48fb-a991-885d22c57425 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:31.652004 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8651ab-c62b-494b-af6f-fe39ccfcee91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:31.652924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8651ab-c62b-494b-af6f-fe39ccfcee91 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:31.653399 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8651ab-c62b-494b-af6f-fe39ccfcee91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:31.653864 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8651ab-c62b-494b-af6f-fe39ccfcee91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:31.654730 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8651ab-c62b-494b-af6f-fe39ccfcee91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:31.657327 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8651ab-c62b-494b-af6f-fe39ccfcee91 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:31.688764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1273/2558] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:20:31 2022] POST /identity/v3/projects => generated 365 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:31.696953 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babd0bf-f65b-4acf-95c8-619ec44d8e12 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:31.706257 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babd0bf-f65b-4acf-95c8-619ec44d8e12 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:31.785443 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1fe68-6ef2-493e-af57-74410dddd266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:31.787376 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1fe68-6ef2-493e-af57-74410dddd266 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:31.787715 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1fe68-6ef2-493e-af57-74410dddd266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:31.788040 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1fe68-6ef2-493e-af57-74410dddd266 admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:31.788794 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f1fe68-6ef2-493e-af57-74410dddd266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:31.790875 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f1fe68-6ef2-493e-af57-74410dddd266 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:31.815759 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1286/2559] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:20:31 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:31.819836 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8651ab-c62b-494b-af6f-fe39ccfcee91 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:31.833171 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8651ab-c62b-494b-af6f-fe39ccfcee91 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:31.838094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252cd59-8260-43f6-980d-8cb43fa2350e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:31.839148 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252cd59-8260-43f6-980d-8cb43fa2350e admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:31.839748 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252cd59-8260-43f6-980d-8cb43fa2350e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:31.839837 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252cd59-8260-43f6-980d-8cb43fa2350e admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:31.840551 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252cd59-8260-43f6-980d-8cb43fa2350e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:31.844082 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252cd59-8260-43f6-980d-8cb43fa2350e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:31.856534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1274/2560] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:31.861469 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1fe68-6ef2-493e-af57-74410dddd266 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:31.873739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1fe68-6ef2-493e-af57-74410dddd266 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:31.878883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91633aa0-c2fa-4b35-9858-d6eaca2278c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:31.879873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91633aa0-c2fa-4b35-9858-d6eaca2278c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:31.880338 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91633aa0-c2fa-4b35-9858-d6eaca2278c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:31.881125 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91633aa0-c2fa-4b35-9858-d6eaca2278c7 admin admin] PATH_INFO: `/v3/projects/fdfee11c6f9546f380dbb8935f6ff06b/users/1a30d243a3604b76bce08ce5aa302765/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:31.901730 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91633aa0-c2fa-4b35-9858-d6eaca2278c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdfee11c6f9546f380dbb8935f6ff06b, user_id=1a30d243a3604b76bce08ce5aa302765, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:31.904568 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91633aa0-c2fa-4b35-9858-d6eaca2278c7 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:31.923571 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1287/2561] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:31 2022] PUT /identity/v3/projects/fdfee11c6f9546f380dbb8935f6ff06b/users/1a30d243a3604b76bce08ce5aa302765/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:31.926040 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252cd59-8260-43f6-980d-8cb43fa2350e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:31.939482 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252cd59-8260-43f6-980d-8cb43fa2350e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:32.032121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db98e73b-62c8-47b2-a9b2-f82319b8803a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:32.034769 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98e73b-62c8-47b2-a9b2-f82319b8803a admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:32.035058 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98e73b-62c8-47b2-a9b2-f82319b8803a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:32.035525 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98e73b-62c8-47b2-a9b2-f82319b8803a admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:32.036276 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db98e73b-62c8-47b2-a9b2-f82319b8803a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:32.037988 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db98e73b-62c8-47b2-a9b2-f82319b8803a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:32.058601 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1275/2562] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:32.059200 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91633aa0-c2fa-4b35-9858-d6eaca2278c7 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:32.069621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:32.072928 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91633aa0-c2fa-4b35-9858-d6eaca2278c7 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:32.084305 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f692f2-9ad5-497c-8314-ead183111d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:32.085514 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f692f2-9ad5-497c-8314-ead183111d1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:32.085514 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f692f2-9ad5-497c-8314-ead183111d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:32.085785 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f692f2-9ad5-497c-8314-ead183111d1e admin admin] PATH_INFO: `/v3/projects/fdfee11c6f9546f380dbb8935f6ff06b/users/1a30d243a3604b76bce08ce5aa302765/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:32.088099 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f692f2-9ad5-497c-8314-ead183111d1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdfee11c6f9546f380dbb8935f6ff06b, user_id=1a30d243a3604b76bce08ce5aa302765, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:32.091181 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f692f2-9ad5-497c-8314-ead183111d1e admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:32.126294 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9ffec-860d-40c7-92f4-801343caed41 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:32.126588 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9ffec-860d-40c7-92f4-801343caed41 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:32.126859 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9ffec-860d-40c7-92f4-801343caed41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:32.128231 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1288/2563] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:32 2022] PUT /identity/v3/projects/fdfee11c6f9546f380dbb8935f6ff06b/users/1a30d243a3604b76bce08ce5aa302765/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:32.159878 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c0b9ffec-860d-40c7-92f4-801343caed41 None None] MFA Rules not processed for user `1a30d243a3604b76bce08ce5aa302765`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:32.162164 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c0b9ffec-860d-40c7-92f4-801343caed41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:32.259369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1276/2564] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:32 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:32.271276 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f692f2-9ad5-497c-8314-ead183111d1e admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:32.273129 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-87f692f2-9ad5-497c-8314-ead183111d1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:32.341404 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f692f2-9ad5-497c-8314-ead183111d1e admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:32.347393 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408b36b-3b37-4a8b-831f-23d960d21f43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:32.348285 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408b36b-3b37-4a8b-831f-23d960d21f43 service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:32.348722 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408b36b-3b37-4a8b-831f-23d960d21f43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:32.349146 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408b36b-3b37-4a8b-831f-23d960d21f43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:32.361487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c408b36b-3b37-4a8b-831f-23d960d21f43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:32.363227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c408b36b-3b37-4a8b-831f-23d960d21f43 service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:32.375503 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1289/2565] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:32 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:20:32.458224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b9ffec-860d-40c7-92f4-801343caed41 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:32.468830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b9ffec-860d-40c7-92f4-801343caed41 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:32.473878 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672b66b-604b-4d12-8bae-24856ccda5c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:32.474786 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672b66b-604b-4d12-8bae-24856ccda5c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:32.475253 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672b66b-604b-4d12-8bae-24856ccda5c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:32.475635 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672b66b-604b-4d12-8bae-24856ccda5c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:32.562076 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:32.584530 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672b66b-604b-4d12-8bae-24856ccda5c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:32.586118 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672b66b-604b-4d12-8bae-24856ccda5c3 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:32.614123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1277/2566] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:32 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:20:35.560041 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408b36b-3b37-4a8b-831f-23d960d21f43 service cinder] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:35.639037 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408b36b-3b37-4a8b-831f-23d960d21f43 service cinder] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:35.644055 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d73b1-c7a8-4333-8568-3b4a5413d83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:35.644945 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d73b1-c7a8-4333-8568-3b4a5413d83e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:35.645233 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d73b1-c7a8-4333-8568-3b4a5413d83e admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:35.645516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d73b1-c7a8-4333-8568-3b4a5413d83e admin admin] PATH_INFO: `/v3/users/83d6910c1cbd488a9c716176998ed5c5` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:35.646903 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d73b1-c7a8-4333-8568-3b4a5413d83e admin admin] RBAC: Authorizing `identity:delete_user(user_id=83d6910c1cbd488a9c716176998ed5c5)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:35.649011 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d73b1-c7a8-4333-8568-3b4a5413d83e admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:35.699202 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-512d73b1-c7a8-4333-8568-3b4a5413d83e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83d6910c1cbd488a9c716176998ed5c5'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:35.708816 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-512d73b1-c7a8-4333-8568-3b4a5413d83e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83d6910c1cbd488a9c716176998ed5c5'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:35.733336 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-512d73b1-c7a8-4333-8568-3b4a5413d83e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83d6910c1cbd488a9c716176998ed5c5'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:35.734175 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-512d73b1-c7a8-4333-8568-3b4a5413d83e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83d6910c1cbd488a9c716176998ed5c5'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:35.743448 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-512d73b1-c7a8-4333-8568-3b4a5413d83e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83d6910c1cbd488a9c716176998ed5c5'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:35.751165 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1290/2567] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:20:35 2022] DELETE /identity/v3/users/83d6910c1cbd488a9c716176998ed5c5 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:35.755798 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a212795-92ac-43fd-b461-32a412b6cca8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:35.756263 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a212795-92ac-43fd-b461-32a412b6cca8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:35.756361 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a212795-92ac-43fd-b461-32a412b6cca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:35.775506 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8a212795-92ac-43fd-b461-32a412b6cca8 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:35.776734 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8a212795-92ac-43fd-b461-32a412b6cca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:35.846808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1278/2568] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:35.856504 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d73b1-c7a8-4333-8568-3b4a5413d83e admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:35.858576 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-512d73b1-c7a8-4333-8568-3b4a5413d83e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:35.873795 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:35.925088 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d73b1-c7a8-4333-8568-3b4a5413d83e admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:35.928941 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00cf59-299c-43fe-9825-8bde841b50e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:35.929598 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00cf59-299c-43fe-9825-8bde841b50e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:35.929889 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00cf59-299c-43fe-9825-8bde841b50e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:35.930196 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00cf59-299c-43fe-9825-8bde841b50e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:35.941080 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00cf59-299c-43fe-9825-8bde841b50e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:35.942585 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00cf59-299c-43fe-9825-8bde841b50e9 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:35.953349 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1291/2569] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:20:36.279681 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a212795-92ac-43fd-b461-32a412b6cca8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:36.282259 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8a212795-92ac-43fd-b461-32a412b6cca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:36.295894 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a212795-92ac-43fd-b461-32a412b6cca8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:36.301193 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146bd4eb-9cf1-4780-8d6e-a2db33fea2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:36.302173 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bd4eb-9cf1-4780-8d6e-a2db33fea2f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:36.302379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bd4eb-9cf1-4780-8d6e-a2db33fea2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:36.302658 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bd4eb-9cf1-4780-8d6e-a2db33fea2f8 admin admin] PATH_INFO: `/v3/projects/40cbc41c87954c02a7f3a27e60dafc65` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:36.304403 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146bd4eb-9cf1-4780-8d6e-a2db33fea2f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40cbc41c87954c02a7f3a27e60dafc65)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:36.315401 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146bd4eb-9cf1-4780-8d6e-a2db33fea2f8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:36.376293 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-146bd4eb-9cf1-4780-8d6e-a2db33fea2f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40cbc41c87954c02a7f3a27e60dafc65', 'request_id': 'req-146bd4eb-9cf1-4780-8d6e-a2db33fea2f8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:36.382356 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-146bd4eb-9cf1-4780-8d6e-a2db33fea2f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40cbc41c87954c02a7f3a27e60dafc65', 'request_id': 'req-146bd4eb-9cf1-4780-8d6e-a2db33fea2f8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:36.394256 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-146bd4eb-9cf1-4780-8d6e-a2db33fea2f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40cbc41c87954c02a7f3a27e60dafc65', 'request_id': 'req-146bd4eb-9cf1-4780-8d6e-a2db33fea2f8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:36.407595 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-146bd4eb-9cf1-4780-8d6e-a2db33fea2f8 admin admin] The token cache is being invalidate because project 40cbc41c87954c02a7f3a27e60dafc65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:20:36.407886 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-146bd4eb-9cf1-4780-8d6e-a2db33fea2f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:36.408543 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-146bd4eb-9cf1-4780-8d6e-a2db33fea2f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:36.410352 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1279/2570] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:20:36 2022] DELETE /identity/v3/projects/40cbc41c87954c02a7f3a27e60dafc65 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:20:36.414636 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00cf59-299c-43fe-9825-8bde841b50e9 service neutron] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:36.425368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00cf59-299c-43fe-9825-8bde841b50e9 service neutron] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:36.486852 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfdf220-3a88-4c2b-a807-0081675a6df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:36.487743 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdf220-3a88-4c2b-a807-0081675a6df1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:36.488035 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdf220-3a88-4c2b-a807-0081675a6df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:36.488314 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdf220-3a88-4c2b-a807-0081675a6df1 admin admin] PATH_INFO: `/v3/users/1a30d243a3604b76bce08ce5aa302765` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:36.489533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfdf220-3a88-4c2b-a807-0081675a6df1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a30d243a3604b76bce08ce5aa302765)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:36.491526 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfdf220-3a88-4c2b-a807-0081675a6df1 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:36.544269 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7dfdf220-3a88-4c2b-a807-0081675a6df1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a30d243a3604b76bce08ce5aa302765'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:36.552850 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7dfdf220-3a88-4c2b-a807-0081675a6df1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a30d243a3604b76bce08ce5aa302765'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:36.564876 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7dfdf220-3a88-4c2b-a807-0081675a6df1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a30d243a3604b76bce08ce5aa302765'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:36.565588 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7dfdf220-3a88-4c2b-a807-0081675a6df1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a30d243a3604b76bce08ce5aa302765'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:36.573619 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-7dfdf220-3a88-4c2b-a807-0081675a6df1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a30d243a3604b76bce08ce5aa302765'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:36.580786 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1292/2571] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:20:36 2022] DELETE /identity/v3/users/1a30d243a3604b76bce08ce5aa302765 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:36.910747 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146bd4eb-9cf1-4780-8d6e-a2db33fea2f8 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:36.912462 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-146bd4eb-9cf1-4780-8d6e-a2db33fea2f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:36.926652 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146bd4eb-9cf1-4780-8d6e-a2db33fea2f8 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:36.991674 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050112c4-7491-4e4b-b6ae-2a7a6204578c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:36.992577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050112c4-7491-4e4b-b6ae-2a7a6204578c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:36.992858 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050112c4-7491-4e4b-b6ae-2a7a6204578c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:36.993131 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050112c4-7491-4e4b-b6ae-2a7a6204578c admin admin] PATH_INFO: `/v3/projects/fdfee11c6f9546f380dbb8935f6ff06b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:36.994401 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050112c4-7491-4e4b-b6ae-2a7a6204578c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdfee11c6f9546f380dbb8935f6ff06b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:36.996313 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050112c4-7491-4e4b-b6ae-2a7a6204578c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:37.015086 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:37.052405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-050112c4-7491-4e4b-b6ae-2a7a6204578c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdfee11c6f9546f380dbb8935f6ff06b', 'request_id': 'req-050112c4-7491-4e4b-b6ae-2a7a6204578c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:37.059535 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-050112c4-7491-4e4b-b6ae-2a7a6204578c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdfee11c6f9546f380dbb8935f6ff06b', 'request_id': 'req-050112c4-7491-4e4b-b6ae-2a7a6204578c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:37.069603 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-050112c4-7491-4e4b-b6ae-2a7a6204578c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdfee11c6f9546f380dbb8935f6ff06b', 'request_id': 'req-050112c4-7491-4e4b-b6ae-2a7a6204578c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:37.083823 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-050112c4-7491-4e4b-b6ae-2a7a6204578c admin admin] The token cache is being invalidate because project fdfee11c6f9546f380dbb8935f6ff06b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:20:37.084115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-050112c4-7491-4e4b-b6ae-2a7a6204578c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:37.084783 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-050112c4-7491-4e4b-b6ae-2a7a6204578c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:37.086501 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1280/2572] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:20:36 2022] DELETE /identity/v3/projects/fdfee11c6f9546f380dbb8935f6ff06b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:37.091636 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1afd7e-9984-4938-9b6e-79654a496e2a None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:37.091925 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1afd7e-9984-4938-9b6e-79654a496e2a None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:37.092857 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1afd7e-9984-4938-9b6e-79654a496e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:37.110615 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-bf1afd7e-9984-4938-9b6e-79654a496e2a None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:37.111856 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-bf1afd7e-9984-4938-9b6e-79654a496e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:37.145785 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:37.177018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1293/2573] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:20:37.185052 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c0935-4a8d-4f35-838d-0698f6e6d3c0 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:37.185348 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c0935-4a8d-4f35-838d-0698f6e6d3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:37.185610 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c0935-4a8d-4f35-838d-0698f6e6d3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:37.205366 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8b9c0935-4a8d-4f35-838d-0698f6e6d3c0 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:37.206618 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8b9c0935-4a8d-4f35-838d-0698f6e6d3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:37.214671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1281/2574] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:37.219690 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1afd7e-9984-4938-9b6e-79654a496e2a None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:37.241307 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1afd7e-9984-4938-9b6e-79654a496e2a None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:37.259864 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f4e21-14f4-491e-abff-fb8bb5b3426f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:37.260691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f4e21-14f4-491e-abff-fb8bb5b3426f admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:37.260975 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f4e21-14f4-491e-abff-fb8bb5b3426f admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:37.261267 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f4e21-14f4-491e-abff-fb8bb5b3426f admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:37.262040 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f4e21-14f4-491e-abff-fb8bb5b3426f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:20:37.262361 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f4e21-14f4-491e-abff-fb8bb5b3426f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:37.264045 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f4e21-14f4-491e-abff-fb8bb5b3426f admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:37.278034 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1294/2575] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:20:37.282578 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c0935-4a8d-4f35-838d-0698f6e6d3c0 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:37.292969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c0935-4a8d-4f35-838d-0698f6e6d3c0 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:37.298255 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd968c-f59a-4f61-b4b4-335d5a48af2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:37.298939 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd968c-f59a-4f61-b4b4-335d5a48af2f admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:37.299222 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd968c-f59a-4f61-b4b4-335d5a48af2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:37.299499 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd968c-f59a-4f61-b4b4-335d5a48af2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:37.300213 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dd968c-f59a-4f61-b4b4-335d5a48af2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:37.302176 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dd968c-f59a-4f61-b4b4-335d5a48af2f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:37.337388 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1282/2576] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:20:37 2022] POST /identity/v3/projects => generated 369 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:37.341681 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f4e21-14f4-491e-abff-fb8bb5b3426f admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:37.353878 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f4e21-14f4-491e-abff-fb8bb5b3426f admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:37.420157 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741838e-e099-4207-a041-b81dbda240a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:37.420887 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741838e-e099-4207-a041-b81dbda240a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:37.421213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741838e-e099-4207-a041-b81dbda240a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:37.421599 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741838e-e099-4207-a041-b81dbda240a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:37.422624 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b741838e-e099-4207-a041-b81dbda240a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:37.427306 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b741838e-e099-4207-a041-b81dbda240a6 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:37.456387 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1295/2577] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:20:37 2022] POST /identity/v3/users => generated 285 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:37.457584 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd968c-f59a-4f61-b4b4-335d5a48af2f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:37.473153 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd968c-f59a-4f61-b4b4-335d5a48af2f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:37.478345 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423ef5f-d90d-4664-a8d7-706c915b75d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:37.479095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423ef5f-d90d-4664-a8d7-706c915b75d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:37.479444 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423ef5f-d90d-4664-a8d7-706c915b75d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:37.481421 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423ef5f-d90d-4664-a8d7-706c915b75d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:37.481421 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423ef5f-d90d-4664-a8d7-706c915b75d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:37.499558 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423ef5f-d90d-4664-a8d7-706c915b75d6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:37.513727 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1283/2578] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:37.520263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741838e-e099-4207-a041-b81dbda240a6 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:37.531456 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741838e-e099-4207-a041-b81dbda240a6 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:37.537622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35386d9d-5efb-48f6-b459-ab847117c85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:37.539409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35386d9d-5efb-48f6-b459-ab847117c85e admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:37.539409 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35386d9d-5efb-48f6-b459-ab847117c85e admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:37.539722 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35386d9d-5efb-48f6-b459-ab847117c85e admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:37.543760 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35386d9d-5efb-48f6-b459-ab847117c85e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:37.543760 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35386d9d-5efb-48f6-b459-ab847117c85e admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:37.558216 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1296/2579] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:37.560053 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423ef5f-d90d-4664-a8d7-706c915b75d6 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:37.579498 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423ef5f-d90d-4664-a8d7-706c915b75d6 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:37.584590 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd95bf-5ab0-4a14-8ae1-b6786449ce6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:37.585501 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd95bf-5ab0-4a14-8ae1-b6786449ce6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:37.585787 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd95bf-5ab0-4a14-8ae1-b6786449ce6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:37.586105 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd95bf-5ab0-4a14-8ae1-b6786449ce6b admin admin] PATH_INFO: `/v3/projects/5725a7a7a22a4652a970c717f51471c6/users/a64281e16b8048fa905bc64e9682e044/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:37.604957 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edd95bf-5ab0-4a14-8ae1-b6786449ce6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5725a7a7a22a4652a970c717f51471c6, user_id=a64281e16b8048fa905bc64e9682e044, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:37.608558 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edd95bf-5ab0-4a14-8ae1-b6786449ce6b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:37.630608 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1284/2580] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:37 2022] PUT /identity/v3/projects/5725a7a7a22a4652a970c717f51471c6/users/a64281e16b8048fa905bc64e9682e044/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:37.651770 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb7eb0-c9d4-4eaf-b4a9-1edc176991d4 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:37.652354 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb7eb0-c9d4-4eaf-b4a9-1edc176991d4 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:37.652354 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb7eb0-c9d4-4eaf-b4a9-1edc176991d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:37.695921 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-aedb7eb0-c9d4-4eaf-b4a9-1edc176991d4 None None] MFA Rules not processed for user `a64281e16b8048fa905bc64e9682e044`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:37.697252 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-aedb7eb0-c9d4-4eaf-b4a9-1edc176991d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:37.784512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1297/2581] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:37 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:37.789985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd95bf-5ab0-4a14-8ae1-b6786449ce6b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:37.875822 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd95bf-5ab0-4a14-8ae1-b6786449ce6b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:37.883451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b61391-2e3a-498b-8106-bf1635c7c156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:37.884418 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b61391-2e3a-498b-8106-bf1635c7c156 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:37.884722 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b61391-2e3a-498b-8106-bf1635c7c156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:37.884974 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b61391-2e3a-498b-8106-bf1635c7c156 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:37.885708 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b61391-2e3a-498b-8106-bf1635c7c156 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:37.887727 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b61391-2e3a-498b-8106-bf1635c7c156 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:37.915125 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1285/2582] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:20:37 2022] POST /identity/v3/projects => generated 367 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:37.921488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb7eb0-c9d4-4eaf-b4a9-1edc176991d4 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:37.934206 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb7eb0-c9d4-4eaf-b4a9-1edc176991d4 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:37.971159 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:38.005986 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e43af1-58dc-46ac-a00a-41afca297c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:38.006699 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e43af1-58dc-46ac-a00a-41afca297c17 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:38.006979 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e43af1-58dc-46ac-a00a-41afca297c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:38.007253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e43af1-58dc-46ac-a00a-41afca297c17 admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:38.007950 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e43af1-58dc-46ac-a00a-41afca297c17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:38.010196 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e43af1-58dc-46ac-a00a-41afca297c17 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:38.038043 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1298/2583] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:20:37 2022] POST /identity/v3/users => generated 290 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:20:38.042160 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b61391-2e3a-498b-8106-bf1635c7c156 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:38.053663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b61391-2e3a-498b-8106-bf1635c7c156 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:38.059308 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57970d41-f433-4ce5-9559-8973c6fd40d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:38.059980 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57970d41-f433-4ce5-9559-8973c6fd40d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:38.060261 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57970d41-f433-4ce5-9559-8973c6fd40d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:38.060542 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57970d41-f433-4ce5-9559-8973c6fd40d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:38.061403 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57970d41-f433-4ce5-9559-8973c6fd40d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:38.063223 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57970d41-f433-4ce5-9559-8973c6fd40d8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:38.078951 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1286/2584] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:20:38.083285 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e43af1-58dc-46ac-a00a-41afca297c17 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:38.095246 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e43af1-58dc-46ac-a00a-41afca297c17 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:38.100670 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b5241-d266-4d0e-8581-fa02dd4869f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:38.101617 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b5241-d266-4d0e-8581-fa02dd4869f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:38.101976 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b5241-d266-4d0e-8581-fa02dd4869f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:38.102301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b5241-d266-4d0e-8581-fa02dd4869f9 admin admin] PATH_INFO: `/v3/projects/dff983309f4e4bf99192a82ac1136c2c/users/731ad8bcb7574840852bd0104155a935/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:38.122316 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714b5241-d266-4d0e-8581-fa02dd4869f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dff983309f4e4bf99192a82ac1136c2c, user_id=731ad8bcb7574840852bd0104155a935, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:38.125351 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714b5241-d266-4d0e-8581-fa02dd4869f9 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:38.142774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1299/2585] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:38 2022] PUT /identity/v3/projects/dff983309f4e4bf99192a82ac1136c2c/users/731ad8bcb7574840852bd0104155a935/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:38.148402 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57970d41-f433-4ce5-9559-8973c6fd40d8 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:38.159674 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57970d41-f433-4ce5-9559-8973c6fd40d8 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:38.170881 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:38.228635 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad47cac-4209-46ad-a66e-381d60050786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:38.229344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad47cac-4209-46ad-a66e-381d60050786 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:38.229631 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad47cac-4209-46ad-a66e-381d60050786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:38.230021 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad47cac-4209-46ad-a66e-381d60050786 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:38.230772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad47cac-4209-46ad-a66e-381d60050786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:38.232432 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad47cac-4209-46ad-a66e-381d60050786 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:38.244700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1287/2586] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:38.250848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b5241-d266-4d0e-8581-fa02dd4869f9 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:38.265620 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b5241-d266-4d0e-8581-fa02dd4869f9 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:38.270824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf978535-10dc-4931-8a59-25bd1ac98b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:38.271836 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf978535-10dc-4931-8a59-25bd1ac98b3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:38.272249 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf978535-10dc-4931-8a59-25bd1ac98b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:38.272622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf978535-10dc-4931-8a59-25bd1ac98b3d admin admin] PATH_INFO: `/v3/projects/dff983309f4e4bf99192a82ac1136c2c/users/731ad8bcb7574840852bd0104155a935/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:38.274978 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf978535-10dc-4931-8a59-25bd1ac98b3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dff983309f4e4bf99192a82ac1136c2c, user_id=731ad8bcb7574840852bd0104155a935, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:38.277738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf978535-10dc-4931-8a59-25bd1ac98b3d admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:38.290448 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1300/2587] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:38 2022] PUT /identity/v3/projects/dff983309f4e4bf99192a82ac1136c2c/users/731ad8bcb7574840852bd0104155a935/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:38.302559 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155b8bb-b945-4d35-9536-1bf01a37a7bf None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:38.302844 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155b8bb-b945-4d35-9536-1bf01a37a7bf None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:38.303150 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155b8bb-b945-4d35-9536-1bf01a37a7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:38.334372 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c155b8bb-b945-4d35-9536-1bf01a37a7bf None None] MFA Rules not processed for user `731ad8bcb7574840852bd0104155a935`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:38.335767 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c155b8bb-b945-4d35-9536-1bf01a37a7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:38.419880 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1288/2588] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:38 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:20:38.432499 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf978535-10dc-4931-8a59-25bd1ac98b3d admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:38.434956 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-bf978535-10dc-4931-8a59-25bd1ac98b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:38.504446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf978535-10dc-4931-8a59-25bd1ac98b3d admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:38.508587 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519b280-2eab-436e-8ecb-e8db47594c96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:38.509516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519b280-2eab-436e-8ecb-e8db47594c96 service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:38.509812 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519b280-2eab-436e-8ecb-e8db47594c96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:38.510128 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519b280-2eab-436e-8ecb-e8db47594c96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:38.520082 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3519b280-2eab-436e-8ecb-e8db47594c96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:38.521563 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3519b280-2eab-436e-8ecb-e8db47594c96 service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:38.532476 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1301/2589] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:38 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:20:40.271408 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155b8bb-b945-4d35-9536-1bf01a37a7bf None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:40.339251 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155b8bb-b945-4d35-9536-1bf01a37a7bf None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:40.346162 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18babd5-0284-4cae-8f6b-f24a376cf010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:40.347567 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18babd5-0284-4cae-8f6b-f24a376cf010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:40.348018 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18babd5-0284-4cae-8f6b-f24a376cf010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:40.348493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18babd5-0284-4cae-8f6b-f24a376cf010 admin admin] PATH_INFO: `/v3/users/a64281e16b8048fa905bc64e9682e044` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:40.350488 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18babd5-0284-4cae-8f6b-f24a376cf010 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a64281e16b8048fa905bc64e9682e044)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:40.353976 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18babd5-0284-4cae-8f6b-f24a376cf010 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:40.420432 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:40.442074 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a18babd5-0284-4cae-8f6b-f24a376cf010 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a64281e16b8048fa905bc64e9682e044'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:40.459456 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a18babd5-0284-4cae-8f6b-f24a376cf010 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a64281e16b8048fa905bc64e9682e044'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:40.469394 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a18babd5-0284-4cae-8f6b-f24a376cf010 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a64281e16b8048fa905bc64e9682e044'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:40.478485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a18babd5-0284-4cae-8f6b-f24a376cf010 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a64281e16b8048fa905bc64e9682e044'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:40.479551 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a18babd5-0284-4cae-8f6b-f24a376cf010 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a64281e16b8048fa905bc64e9682e044'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:40.489287 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1289/2590] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:20:40 2022] DELETE /identity/v3/users/a64281e16b8048fa905bc64e9682e044 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:40.494630 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea377b-14ac-4d1a-8e86-01aabcd3fb8c None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:40.495158 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea377b-14ac-4d1a-8e86-01aabcd3fb8c None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:40.495276 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea377b-14ac-4d1a-8e86-01aabcd3fb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:40.516931 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-44ea377b-14ac-4d1a-8e86-01aabcd3fb8c None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:40.518344 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-44ea377b-14ac-4d1a-8e86-01aabcd3fb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:40.596134 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1302/2591] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:20:40.606201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18babd5-0284-4cae-8f6b-f24a376cf010 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:40.608445 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a18babd5-0284-4cae-8f6b-f24a376cf010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:40.687851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18babd5-0284-4cae-8f6b-f24a376cf010 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:40.691547 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d4fc9-777e-48e4-b833-08c80f031ead service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:40.692203 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d4fc9-777e-48e4-b833-08c80f031ead service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:40.692484 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d4fc9-777e-48e4-b833-08c80f031ead service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:40.692754 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d4fc9-777e-48e4-b833-08c80f031ead service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:40.703551 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d4fc9-777e-48e4-b833-08c80f031ead service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:40.705109 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d4fc9-777e-48e4-b833-08c80f031ead service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:40.715106 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1290/2592] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:20:41.106055 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea377b-14ac-4d1a-8e86-01aabcd3fb8c None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:41.109261 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-44ea377b-14ac-4d1a-8e86-01aabcd3fb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:41.125422 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea377b-14ac-4d1a-8e86-01aabcd3fb8c None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:41.130786 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5dac6-7a28-4f60-bb08-1936428b2291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:41.132794 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5dac6-7a28-4f60-bb08-1936428b2291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:41.132794 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5dac6-7a28-4f60-bb08-1936428b2291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:41.132794 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5dac6-7a28-4f60-bb08-1936428b2291 admin admin] PATH_INFO: `/v3/projects/5725a7a7a22a4652a970c717f51471c6` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:41.134976 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f5dac6-7a28-4f60-bb08-1936428b2291 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5725a7a7a22a4652a970c717f51471c6)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:41.137178 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f5dac6-7a28-4f60-bb08-1936428b2291 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:41.168736 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:41.217366 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-50f5dac6-7a28-4f60-bb08-1936428b2291 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5725a7a7a22a4652a970c717f51471c6', 'request_id': 'req-50f5dac6-7a28-4f60-bb08-1936428b2291'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:41.223860 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-50f5dac6-7a28-4f60-bb08-1936428b2291 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5725a7a7a22a4652a970c717f51471c6', 'request_id': 'req-50f5dac6-7a28-4f60-bb08-1936428b2291'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:41.233324 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-50f5dac6-7a28-4f60-bb08-1936428b2291 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5725a7a7a22a4652a970c717f51471c6', 'request_id': 'req-50f5dac6-7a28-4f60-bb08-1936428b2291'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:41.247997 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-50f5dac6-7a28-4f60-bb08-1936428b2291 admin admin] The token cache is being invalidate because project 5725a7a7a22a4652a970c717f51471c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:20:41.247997 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-50f5dac6-7a28-4f60-bb08-1936428b2291 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:41.247997 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-50f5dac6-7a28-4f60-bb08-1936428b2291 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:41.249881 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1303/2593] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:20:41 2022] DELETE /identity/v3/projects/5725a7a7a22a4652a970c717f51471c6 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:41.254363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d4fc9-777e-48e4-b833-08c80f031ead service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:41.266639 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d4fc9-777e-48e4-b833-08c80f031ead service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:41.331767 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc87ff-9528-4550-bb88-89744469ff6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:41.332666 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc87ff-9528-4550-bb88-89744469ff6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:41.332949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc87ff-9528-4550-bb88-89744469ff6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:41.333229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc87ff-9528-4550-bb88-89744469ff6a admin admin] PATH_INFO: `/v3/users/731ad8bcb7574840852bd0104155a935` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:41.334473 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc87ff-9528-4550-bb88-89744469ff6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=731ad8bcb7574840852bd0104155a935)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:41.336428 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc87ff-9528-4550-bb88-89744469ff6a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:41.425043 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-45fc87ff-9528-4550-bb88-89744469ff6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '731ad8bcb7574840852bd0104155a935'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:41.437142 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:41.444730 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-45fc87ff-9528-4550-bb88-89744469ff6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '731ad8bcb7574840852bd0104155a935'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:41.456700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-45fc87ff-9528-4550-bb88-89744469ff6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '731ad8bcb7574840852bd0104155a935'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:41.470830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-45fc87ff-9528-4550-bb88-89744469ff6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '731ad8bcb7574840852bd0104155a935'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:41.472179 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-45fc87ff-9528-4550-bb88-89744469ff6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '731ad8bcb7574840852bd0104155a935'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:41.479977 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1291/2594] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:20:41 2022] DELETE /identity/v3/users/731ad8bcb7574840852bd0104155a935 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:41.960651 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5dac6-7a28-4f60-bb08-1936428b2291 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:41.964110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-50f5dac6-7a28-4f60-bb08-1936428b2291 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:41.983680 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5dac6-7a28-4f60-bb08-1936428b2291 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:42.055886 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb850323-3a85-4711-ba39-ac0ef389e050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:42.057066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb850323-3a85-4711-ba39-ac0ef389e050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:42.057384 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb850323-3a85-4711-ba39-ac0ef389e050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:42.057665 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb850323-3a85-4711-ba39-ac0ef389e050 admin admin] PATH_INFO: `/v3/projects/dff983309f4e4bf99192a82ac1136c2c` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:42.059066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb850323-3a85-4711-ba39-ac0ef389e050 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dff983309f4e4bf99192a82ac1136c2c)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:42.061287 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb850323-3a85-4711-ba39-ac0ef389e050 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:42.123784 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-fb850323-3a85-4711-ba39-ac0ef389e050 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dff983309f4e4bf99192a82ac1136c2c', 'request_id': 'req-fb850323-3a85-4711-ba39-ac0ef389e050'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:42.130331 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-fb850323-3a85-4711-ba39-ac0ef389e050 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dff983309f4e4bf99192a82ac1136c2c', 'request_id': 'req-fb850323-3a85-4711-ba39-ac0ef389e050'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:42.140431 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-fb850323-3a85-4711-ba39-ac0ef389e050 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dff983309f4e4bf99192a82ac1136c2c', 'request_id': 'req-fb850323-3a85-4711-ba39-ac0ef389e050'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:42.154415 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-fb850323-3a85-4711-ba39-ac0ef389e050 admin admin] The token cache is being invalidate because project dff983309f4e4bf99192a82ac1136c2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:20:42.155493 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-fb850323-3a85-4711-ba39-ac0ef389e050 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:42.156212 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-fb850323-3a85-4711-ba39-ac0ef389e050 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:42.158017 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1304/2595] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:20:41 2022] DELETE /identity/v3/projects/dff983309f4e4bf99192a82ac1136c2c => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:20:42.163070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db251854-a118-4f42-ba7a-8674f047b53b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:42.163395 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db251854-a118-4f42-ba7a-8674f047b53b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:42.163677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db251854-a118-4f42-ba7a-8674f047b53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:42.183413 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-db251854-a118-4f42-ba7a-8674f047b53b None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:42.184652 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-db251854-a118-4f42-ba7a-8674f047b53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:42.256458 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1292/2596] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:20:42.266111 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd03ff-17df-430b-be2e-f153a631e11e None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:42.266405 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd03ff-17df-430b-be2e-f153a631e11e None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:42.267535 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd03ff-17df-430b-be2e-f153a631e11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:42.288939 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-c2dd03ff-17df-430b-be2e-f153a631e11e None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:42.290221 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-c2dd03ff-17df-430b-be2e-f153a631e11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:42.299393 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1305/2597] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:20:42.305482 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db251854-a118-4f42-ba7a-8674f047b53b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:42.315832 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db251854-a118-4f42-ba7a-8674f047b53b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:42.323081 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5adfcc3-8fdc-4c51-b073-51386c5bc6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:42.323779 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5adfcc3-8fdc-4c51-b073-51386c5bc6db admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:42.324062 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5adfcc3-8fdc-4c51-b073-51386c5bc6db admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:42.324340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5adfcc3-8fdc-4c51-b073-51386c5bc6db admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:42.325093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5adfcc3-8fdc-4c51-b073-51386c5bc6db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:20:42.325399 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5adfcc3-8fdc-4c51-b073-51386c5bc6db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:42.327907 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5adfcc3-8fdc-4c51-b073-51386c5bc6db admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:42.343378 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1293/2598] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:20:42.349393 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd03ff-17df-430b-be2e-f153a631e11e None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:42.359703 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd03ff-17df-430b-be2e-f153a631e11e None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:42.365135 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d467d7-1afe-4423-9fef-4b54d70cb87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:42.365795 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d467d7-1afe-4423-9fef-4b54d70cb87c admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:42.366107 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d467d7-1afe-4423-9fef-4b54d70cb87c admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:42.366386 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d467d7-1afe-4423-9fef-4b54d70cb87c admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:42.367102 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d467d7-1afe-4423-9fef-4b54d70cb87c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:42.369142 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d467d7-1afe-4423-9fef-4b54d70cb87c admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:42.401991 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1306/2599] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:20:42 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:20:42.407674 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5adfcc3-8fdc-4c51-b073-51386c5bc6db admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:42.421134 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5adfcc3-8fdc-4c51-b073-51386c5bc6db admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:42.522422 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc47bd-d09c-468f-82b6-57edaf231500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:42.525977 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc47bd-d09c-468f-82b6-57edaf231500 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:42.525977 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc47bd-d09c-468f-82b6-57edaf231500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:42.525977 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc47bd-d09c-468f-82b6-57edaf231500 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:42.526304 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc47bd-d09c-468f-82b6-57edaf231500 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:42.529951 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc47bd-d09c-468f-82b6-57edaf231500 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:42.570050 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d467d7-1afe-4423-9fef-4b54d70cb87c admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:42.574116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1294/2600] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:20:42 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:42.591676 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d467d7-1afe-4423-9fef-4b54d70cb87c admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:42.601431 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a92bf-de94-44e1-aa59-3a9639c687b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:42.602707 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a92bf-de94-44e1-aa59-3a9639c687b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:42.603221 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a92bf-de94-44e1-aa59-3a9639c687b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:42.603750 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a92bf-de94-44e1-aa59-3a9639c687b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:42.605382 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8a92bf-de94-44e1-aa59-3a9639c687b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:42.609590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8a92bf-de94-44e1-aa59-3a9639c687b0 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:42.620044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:42.645977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1307/2601] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:42.646428 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc47bd-d09c-468f-82b6-57edaf231500 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:42.664321 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc47bd-d09c-468f-82b6-57edaf231500 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:42.674142 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8ad27-8c77-40f2-bb4f-4533b3336335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:42.675509 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8ad27-8c77-40f2-bb4f-4533b3336335 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:42.675990 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8ad27-8c77-40f2-bb4f-4533b3336335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:42.676294 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8ad27-8c77-40f2-bb4f-4533b3336335 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:42.677092 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8ad27-8c77-40f2-bb4f-4533b3336335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:42.678750 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8ad27-8c77-40f2-bb4f-4533b3336335 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:42.691931 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1295/2602] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:42.696156 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a92bf-de94-44e1-aa59-3a9639c687b0 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:42.707337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a92bf-de94-44e1-aa59-3a9639c687b0 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:42.712622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cbdf00-b92a-4de5-9c79-0ed5895a6084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:42.713521 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbdf00-b92a-4de5-9c79-0ed5895a6084 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:42.713808 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbdf00-b92a-4de5-9c79-0ed5895a6084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:42.714123 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbdf00-b92a-4de5-9c79-0ed5895a6084 admin admin] PATH_INFO: `/v3/projects/846e94445a5f4ad78208ae5e0dad8ea0/users/bcd37d313d4445228443b8c56700a9c1/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:42.730182 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cbdf00-b92a-4de5-9c79-0ed5895a6084 admin admin] RBAC: Authorizing `identity:create_grant(project_id=846e94445a5f4ad78208ae5e0dad8ea0, user_id=bcd37d313d4445228443b8c56700a9c1, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:42.733185 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cbdf00-b92a-4de5-9c79-0ed5895a6084 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:42.752483 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1308/2603] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:42 2022] PUT /identity/v3/projects/846e94445a5f4ad78208ae5e0dad8ea0/users/bcd37d313d4445228443b8c56700a9c1/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:42.764705 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7dedc-363a-4df0-9afe-9fe28f1ae422 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:42.764925 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7dedc-363a-4df0-9afe-9fe28f1ae422 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:42.765205 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7dedc-363a-4df0-9afe-9fe28f1ae422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:42.811741 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-19d7dedc-363a-4df0-9afe-9fe28f1ae422 None None] MFA Rules not processed for user `bcd37d313d4445228443b8c56700a9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:42.813145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-19d7dedc-363a-4df0-9afe-9fe28f1ae422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:42.855535 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:42.900069 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1296/2604] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:42.902341 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e4cfe-fe84-40d0-a121-a2cb9e1d2ba7 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:42.902678 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e4cfe-fe84-40d0-a121-a2cb9e1d2ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:42.902922 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e4cfe-fe84-40d0-a121-a2cb9e1d2ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:42.905992 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d7dedc-363a-4df0-9afe-9fe28f1ae422 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:42.921798 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-872e4cfe-fe84-40d0-a121-a2cb9e1d2ba7 None None] MFA Rules not processed for user `2c2e0e19880f4b048faf034dd38c7365`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:42.923073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-872e4cfe-fe84-40d0-a121-a2cb9e1d2ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:42.976510 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d7dedc-363a-4df0-9afe-9fe28f1ae422 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:42.981352 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47520f9a-0755-409a-b77f-39d61ec8fdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:42.982196 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47520f9a-0755-409a-b77f-39d61ec8fdbb admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:42.982444 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47520f9a-0755-409a-b77f-39d61ec8fdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:42.982897 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47520f9a-0755-409a-b77f-39d61ec8fdbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:42.983968 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1309/2605] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 16:20:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:42.984594 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47520f9a-0755-409a-b77f-39d61ec8fdbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:42.986385 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47520f9a-0755-409a-b77f-39d61ec8fdbb admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:43.018934 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1297/2606] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:20:42 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:20:43.024824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e4cfe-fe84-40d0-a121-a2cb9e1d2ba7 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:43.038353 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47520f9a-0755-409a-b77f-39d61ec8fdbb admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:43.040066 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-47520f9a-0755-409a-b77f-39d61ec8fdbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:43.042360 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e4cfe-fe84-40d0-a121-a2cb9e1d2ba7 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:43.120345 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47520f9a-0755-409a-b77f-39d61ec8fdbb admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:43.126174 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802dfa0-253c-4b90-8b67-2f0c8ba2a7a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0082e37ce54f49f7837601349508c89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:43.126811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802dfa0-253c-4b90-8b67-2f0c8ba2a7a4 service swift] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:43.127092 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802dfa0-253c-4b90-8b67-2f0c8ba2a7a4 service swift] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:43.127372 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802dfa0-253c-4b90-8b67-2f0c8ba2a7a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:43.127750 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a929a2d-d976-4501-b427-8d11c2b37074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:43.128446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a929a2d-d976-4501-b427-8d11c2b37074 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:43.128731 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a929a2d-d976-4501-b427-8d11c2b37074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:43.129007 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a929a2d-d976-4501-b427-8d11c2b37074 admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:43.129726 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a929a2d-d976-4501-b427-8d11c2b37074 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:43.131736 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a929a2d-d976-4501-b427-8d11c2b37074 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:43.159005 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1310/2607] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:20:43 2022] POST /identity/v3/users => generated 300 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:43.160668 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a929a2d-d976-4501-b427-8d11c2b37074 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:43.174743 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a929a2d-d976-4501-b427-8d11c2b37074 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:43.180446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7585f9-e24b-4285-a05a-915d42e6b525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:43.181136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7585f9-e24b-4285-a05a-915d42e6b525 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:43.181420 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7585f9-e24b-4285-a05a-915d42e6b525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:43.181699 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7585f9-e24b-4285-a05a-915d42e6b525 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:43.182567 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7585f9-e24b-4285-a05a-915d42e6b525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:43.184477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7585f9-e24b-4285-a05a-915d42e6b525 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:43.197389 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1311/2608] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:43.203987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7585f9-e24b-4285-a05a-915d42e6b525 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:43.205306 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802dfa0-253c-4b90-8b67-2f0c8ba2a7a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:43.206977 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802dfa0-253c-4b90-8b67-2f0c8ba2a7a4 service swift] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:43.210839 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:43.218381 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7585f9-e24b-4285-a05a-915d42e6b525 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:43.219623 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1298/2609] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 16:20:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 16:20:43.225192 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ba5f1-6895-4b23-8827-410382cb582a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:43.226068 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ba5f1-6895-4b23-8827-410382cb582a admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:43.226453 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ba5f1-6895-4b23-8827-410382cb582a admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:43.226666 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ba5f1-6895-4b23-8827-410382cb582a admin admin] PATH_INFO: `/v3/projects/62e988de91f24340b2788ca5eb8000bf/users/dbf1662e29c54781ad9543d820f74b24/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:43.250118 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ba5f1-6895-4b23-8827-410382cb582a admin admin] RBAC: Authorizing `identity:create_grant(project_id=62e988de91f24340b2788ca5eb8000bf, user_id=dbf1662e29c54781ad9543d820f74b24, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:43.252475 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ba5f1-6895-4b23-8827-410382cb582a admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:43.270141 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1312/2610] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:43 2022] PUT /identity/v3/projects/62e988de91f24340b2788ca5eb8000bf/users/dbf1662e29c54781ad9543d820f74b24/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:20:43.274216 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802dfa0-253c-4b90-8b67-2f0c8ba2a7a4 service swift] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:43.285859 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802dfa0-253c-4b90-8b67-2f0c8ba2a7a4 service swift] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:43.351647 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dad049-34f9-48cb-ac32-b9b2c56e51a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:43.352442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dad049-34f9-48cb-ac32-b9b2c56e51a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:43.352711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dad049-34f9-48cb-ac32-b9b2c56e51a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:43.352985 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dad049-34f9-48cb-ac32-b9b2c56e51a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:43.353778 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dad049-34f9-48cb-ac32-b9b2c56e51a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:43.355822 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dad049-34f9-48cb-ac32-b9b2c56e51a6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:43.368523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1299/2611] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:43.372870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ba5f1-6895-4b23-8827-410382cb582a admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:43.385236 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ba5f1-6895-4b23-8827-410382cb582a admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:43.391103 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f3679-6884-40fc-a3a1-a45d4e55a363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:43.392092 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f3679-6884-40fc-a3a1-a45d4e55a363 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:43.392276 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f3679-6884-40fc-a3a1-a45d4e55a363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:43.393089 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f3679-6884-40fc-a3a1-a45d4e55a363 admin admin] PATH_INFO: `/v3/projects/62e988de91f24340b2788ca5eb8000bf/users/dbf1662e29c54781ad9543d820f74b24/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:43.394709 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8f3679-6884-40fc-a3a1-a45d4e55a363 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62e988de91f24340b2788ca5eb8000bf, user_id=dbf1662e29c54781ad9543d820f74b24, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:43.397437 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8f3679-6884-40fc-a3a1-a45d4e55a363 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:43.409564 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1313/2612] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:43 2022] PUT /identity/v3/projects/62e988de91f24340b2788ca5eb8000bf/users/dbf1662e29c54781ad9543d820f74b24/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:43.419917 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4c021-71ea-4535-bb03-05b72e27a900 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:43.420216 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4c021-71ea-4535-bb03-05b72e27a900 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:43.420502 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4c021-71ea-4535-bb03-05b72e27a900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:43.450766 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-76b4c021-71ea-4535-bb03-05b72e27a900 None None] MFA Rules not processed for user `dbf1662e29c54781ad9543d820f74b24`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:43.453212 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-76b4c021-71ea-4535-bb03-05b72e27a900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:43.530955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1300/2613] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:43 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:20:43.542150 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f3679-6884-40fc-a3a1-a45d4e55a363 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:43.544094 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-3a8f3679-6884-40fc-a3a1-a45d4e55a363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:43.574439 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:43.612515 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f3679-6884-40fc-a3a1-a45d4e55a363 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:43.616302 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212596e-7e4b-4313-8e26-73cfc7c5b6c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:43.617055 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212596e-7e4b-4313-8e26-73cfc7c5b6c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:43.617340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212596e-7e4b-4313-8e26-73cfc7c5b6c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:43.617736 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212596e-7e4b-4313-8e26-73cfc7c5b6c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:43.630007 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3212596e-7e4b-4313-8e26-73cfc7c5b6c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:43.631505 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3212596e-7e4b-4313-8e26-73cfc7c5b6c0 service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:43.642057 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1314/2614] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:20:43.716869 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4c021-71ea-4535-bb03-05b72e27a900 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:43.799509 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cdbc6-4d6d-429f-a58b-d233a21e6074 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:43.799796 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cdbc6-4d6d-429f-a58b-d233a21e6074 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:43.800069 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cdbc6-4d6d-429f-a58b-d233a21e6074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:43.811952 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4c021-71ea-4535-bb03-05b72e27a900 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:43.816643 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf01034-5629-4f07-a6e0-a5b7f95f1cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:43.817535 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf01034-5629-4f07-a6e0-a5b7f95f1cc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:43.817814 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf01034-5629-4f07-a6e0-a5b7f95f1cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:43.818117 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf01034-5629-4f07-a6e0-a5b7f95f1cc1 admin admin] PATH_INFO: `/v3/users/bcd37d313d4445228443b8c56700a9c1` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:43.819357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf01034-5629-4f07-a6e0-a5b7f95f1cc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcd37d313d4445228443b8c56700a9c1)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:43.820652 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-0f3cdbc6-4d6d-429f-a58b-d233a21e6074 None None] MFA Rules not processed for user `2c2e0e19880f4b048faf034dd38c7365`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:43.821292 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf01034-5629-4f07-a6e0-a5b7f95f1cc1 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:43.822038 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-0f3cdbc6-4d6d-429f-a58b-d233a21e6074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:43.851700 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:43.889027 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-acf01034-5629-4f07-a6e0-a5b7f95f1cc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcd37d313d4445228443b8c56700a9c1'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:43.896636 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1315/2615] 10.222.0.17 () {58 vars in 967 bytes} [Thu Jun 16 16:20:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:43.899683 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-acf01034-5629-4f07-a6e0-a5b7f95f1cc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcd37d313d4445228443b8c56700a9c1'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:43.908317 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-acf01034-5629-4f07-a6e0-a5b7f95f1cc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcd37d313d4445228443b8c56700a9c1'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:43.909622 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3cdbc6-4d6d-429f-a58b-d233a21e6074 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:43.916233 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-acf01034-5629-4f07-a6e0-a5b7f95f1cc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcd37d313d4445228443b8c56700a9c1'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:43.916901 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-acf01034-5629-4f07-a6e0-a5b7f95f1cc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcd37d313d4445228443b8c56700a9c1'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:43.923766 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1301/2616] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:20:43 2022] DELETE /identity/v3/users/bcd37d313d4445228443b8c56700a9c1 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:43.928363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fb0da-a2fb-4668-8066-679c25aa4ba9 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:43.928659 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fb0da-a2fb-4668-8066-679c25aa4ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:43.928944 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fb0da-a2fb-4668-8066-679c25aa4ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:43.947857 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-534fb0da-a2fb-4668-8066-679c25aa4ba9 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:43.949071 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-534fb0da-a2fb-4668-8066-679c25aa4ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:43.981405 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3cdbc6-4d6d-429f-a58b-d233a21e6074 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:44.012998 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1302/2617] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:20:44.023035 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534fb0da-a2fb-4668-8066-679c25aa4ba9 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:44.024541 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-534fb0da-a2fb-4668-8066-679c25aa4ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:44.046719 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d92b371-5b78-476d-8345-7260b8976a02 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0082e37ce54f49f7837601349508c89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:44.047377 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92b371-5b78-476d-8345-7260b8976a02 service swift] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:44.047616 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92b371-5b78-476d-8345-7260b8976a02 service swift] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:44.047902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92b371-5b78-476d-8345-7260b8976a02 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:44.049791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-3d92b371-5b78-476d-8345-7260b8976a02 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:44.064465 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:44.087437 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534fb0da-a2fb-4668-8066-679c25aa4ba9 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:44.091418 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d67c0-8609-4148-a1ea-da4e03369505 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:44.092661 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d67c0-8609-4148-a1ea-da4e03369505 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:44.092958 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d67c0-8609-4148-a1ea-da4e03369505 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:44.093234 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d67c0-8609-4148-a1ea-da4e03369505 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:44.102929 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951d67c0-8609-4148-a1ea-da4e03369505 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:44.104374 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951d67c0-8609-4148-a1ea-da4e03369505 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:44.114506 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d92b371-5b78-476d-8345-7260b8976a02 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:44.115503 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1303/2618] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:20:44.118705 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d92b371-5b78-476d-8345-7260b8976a02 service swift] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:44.128002 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1316/2619] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 16:20:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 16:20:44.339570 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3300e-ade4-4f01-97c6-cb8e5403182a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:44.339838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3300e-ade4-4f01-97c6-cb8e5403182a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:44.340116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3300e-ade4-4f01-97c6-cb8e5403182a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:44.358997 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e2c3300e-ade4-4f01-97c6-cb8e5403182a None None] MFA Rules not processed for user `2c2e0e19880f4b048faf034dd38c7365`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:44.360203 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e2c3300e-ade4-4f01-97c6-cb8e5403182a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:44.367742 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1304/2620] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 16:20:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:44.380511 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d92b371-5b78-476d-8345-7260b8976a02 service swift] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:44.382052 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-3d92b371-5b78-476d-8345-7260b8976a02 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:44.397013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d92b371-5b78-476d-8345-7260b8976a02 service swift] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:44.403330 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04817a6-71ae-4229-95d3-eb32faebe1ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0082e37ce54f49f7837601349508c89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:44.403998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04817a6-71ae-4229-95d3-eb32faebe1ad service swift] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:44.404305 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04817a6-71ae-4229-95d3-eb32faebe1ad service swift] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:44.405157 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04817a6-71ae-4229-95d3-eb32faebe1ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:44.415883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04817a6-71ae-4229-95d3-eb32faebe1ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:44.418272 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04817a6-71ae-4229-95d3-eb32faebe1ad service swift] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:44.426438 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3300e-ade4-4f01-97c6-cb8e5403182a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:44.428146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e2c3300e-ade4-4f01-97c6-cb8e5403182a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:44.430364 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1317/2621] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 16:20:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:20:44.460838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3300e-ade4-4f01-97c6-cb8e5403182a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:44.465561 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55e1c7-3809-41b8-a675-9e4c48dae003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:44.466433 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55e1c7-3809-41b8-a675-9e4c48dae003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:44.466719 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55e1c7-3809-41b8-a675-9e4c48dae003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:44.467542 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55e1c7-3809-41b8-a675-9e4c48dae003 admin admin] PATH_INFO: `/v3/projects/846e94445a5f4ad78208ae5e0dad8ea0` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:44.469031 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d55e1c7-3809-41b8-a675-9e4c48dae003 admin admin] RBAC: Authorizing `identity:delete_project(project_id=846e94445a5f4ad78208ae5e0dad8ea0)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:44.471474 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d55e1c7-3809-41b8-a675-9e4c48dae003 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:44.510620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:44.528637 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0d55e1c7-3809-41b8-a675-9e4c48dae003 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '846e94445a5f4ad78208ae5e0dad8ea0', 'request_id': 'req-0d55e1c7-3809-41b8-a675-9e4c48dae003'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:44.535624 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0d55e1c7-3809-41b8-a675-9e4c48dae003 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '846e94445a5f4ad78208ae5e0dad8ea0', 'request_id': 'req-0d55e1c7-3809-41b8-a675-9e4c48dae003'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:44.543552 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0d55e1c7-3809-41b8-a675-9e4c48dae003 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '846e94445a5f4ad78208ae5e0dad8ea0', 'request_id': 'req-0d55e1c7-3809-41b8-a675-9e4c48dae003'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:44.556641 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0d55e1c7-3809-41b8-a675-9e4c48dae003 admin admin] The token cache is being invalidate because project 846e94445a5f4ad78208ae5e0dad8ea0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:20:44.556933 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0d55e1c7-3809-41b8-a675-9e4c48dae003 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:44.557596 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0d55e1c7-3809-41b8-a675-9e4c48dae003 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:44.559167 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1305/2622] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:20:44 2022] DELETE /identity/v3/projects/846e94445a5f4ad78208ae5e0dad8ea0 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:20:44.563434 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04817a6-71ae-4229-95d3-eb32faebe1ad service swift] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:44.575332 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04817a6-71ae-4229-95d3-eb32faebe1ad service swift] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:44.644848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c637d-6c6e-4346-884d-2cc0976adc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:44.645908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c637d-6c6e-4346-884d-2cc0976adc9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:44.646321 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c637d-6c6e-4346-884d-2cc0976adc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:44.646696 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c637d-6c6e-4346-884d-2cc0976adc9c admin admin] PATH_INFO: `/v3/users/dbf1662e29c54781ad9543d820f74b24` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:44.648175 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c637d-6c6e-4346-884d-2cc0976adc9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbf1662e29c54781ad9543d820f74b24)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:44.650636 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c637d-6c6e-4346-884d-2cc0976adc9c admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:44.705753 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-421c637d-6c6e-4346-884d-2cc0976adc9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbf1662e29c54781ad9543d820f74b24'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:44.713201 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-421c637d-6c6e-4346-884d-2cc0976adc9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbf1662e29c54781ad9543d820f74b24'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:44.727461 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-421c637d-6c6e-4346-884d-2cc0976adc9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbf1662e29c54781ad9543d820f74b24'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:44.728204 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-421c637d-6c6e-4346-884d-2cc0976adc9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbf1662e29c54781ad9543d820f74b24'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:44.738094 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-421c637d-6c6e-4346-884d-2cc0976adc9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbf1662e29c54781ad9543d820f74b24'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:44.749267 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1318/2623] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:20:44 2022] DELETE /identity/v3/users/dbf1662e29c54781ad9543d820f74b24 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:44.796621 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470b2b7-36f7-4bce-a9b5-f45b83be7367 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:44.796914 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470b2b7-36f7-4bce-a9b5-f45b83be7367 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:44.797201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470b2b7-36f7-4bce-a9b5-f45b83be7367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:44.816492 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1470b2b7-36f7-4bce-a9b5-f45b83be7367 None None] MFA Rules not processed for user `2c2e0e19880f4b048faf034dd38c7365`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:44.817631 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1470b2b7-36f7-4bce-a9b5-f45b83be7367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:44.881303 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1306/2624] 10.222.0.17 () {58 vars in 967 bytes} [Thu Jun 16 16:20:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:44.894691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c637d-6c6e-4346-884d-2cc0976adc9c admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:44.896192 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-421c637d-6c6e-4346-884d-2cc0976adc9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:44.927694 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:44.987275 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c637d-6c6e-4346-884d-2cc0976adc9c admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:44.993615 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ab1da-e24e-4ab8-b367-cc8e7f10bba6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0082e37ce54f49f7837601349508c89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:44.994321 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ab1da-e24e-4ab8-b367-cc8e7f10bba6 service swift] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:44.994613 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ab1da-e24e-4ab8-b367-cc8e7f10bba6 service swift] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:44.995066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ab1da-e24e-4ab8-b367-cc8e7f10bba6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:45.004887 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ab1da-e24e-4ab8-b367-cc8e7f10bba6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:45.006597 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ab1da-e24e-4ab8-b367-cc8e7f10bba6 service swift] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:45.017541 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1319/2625] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 16:20:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:20:45.088108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470b2b7-36f7-4bce-a9b5-f45b83be7367 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:45.089863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1470b2b7-36f7-4bce-a9b5-f45b83be7367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:45.183720 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470b2b7-36f7-4bce-a9b5-f45b83be7367 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:45.191788 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f4814-96e6-42aa-a4ba-3f3ab579df91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:45.199740 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f4814-96e6-42aa-a4ba-3f3ab579df91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:45.200071 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f4814-96e6-42aa-a4ba-3f3ab579df91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:45.200364 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f4814-96e6-42aa-a4ba-3f3ab579df91 admin admin] PATH_INFO: `/v3/projects/62e988de91f24340b2788ca5eb8000bf` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:45.202256 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385f4814-96e6-42aa-a4ba-3f3ab579df91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62e988de91f24340b2788ca5eb8000bf)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:45.204217 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385f4814-96e6-42aa-a4ba-3f3ab579df91 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:45.282453 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-385f4814-96e6-42aa-a4ba-3f3ab579df91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62e988de91f24340b2788ca5eb8000bf', 'request_id': 'req-385f4814-96e6-42aa-a4ba-3f3ab579df91'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:45.288963 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-385f4814-96e6-42aa-a4ba-3f3ab579df91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62e988de91f24340b2788ca5eb8000bf', 'request_id': 'req-385f4814-96e6-42aa-a4ba-3f3ab579df91'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:45.301356 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-385f4814-96e6-42aa-a4ba-3f3ab579df91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62e988de91f24340b2788ca5eb8000bf', 'request_id': 'req-385f4814-96e6-42aa-a4ba-3f3ab579df91'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:45.303063 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:45.318292 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-385f4814-96e6-42aa-a4ba-3f3ab579df91 admin admin] The token cache is being invalidate because project 62e988de91f24340b2788ca5eb8000bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:20:45.318581 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-385f4814-96e6-42aa-a4ba-3f3ab579df91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:45.319425 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-385f4814-96e6-42aa-a4ba-3f3ab579df91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:45.321069 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1307/2626] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:20:45 2022] DELETE /identity/v3/projects/62e988de91f24340b2788ca5eb8000bf => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:45.329892 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aee442-efcf-4cbc-8b8d-3cea121659ff None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:45.329892 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aee442-efcf-4cbc-8b8d-3cea121659ff None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:45.329892 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aee442-efcf-4cbc-8b8d-3cea121659ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:45.354533 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-b2aee442-efcf-4cbc-8b8d-3cea121659ff None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:45.356176 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-b2aee442-efcf-4cbc-8b8d-3cea121659ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:45.426299 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1320/2627] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:45.434258 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f59aa0-f31c-42a1-8e32-1ed2b2d5da5e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:45.434743 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f59aa0-f31c-42a1-8e32-1ed2b2d5da5e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:45.434842 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f59aa0-f31c-42a1-8e32-1ed2b2d5da5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:45.453706 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d1f59aa0-f31c-42a1-8e32-1ed2b2d5da5e None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:45.454950 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d1f59aa0-f31c-42a1-8e32-1ed2b2d5da5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:45.463136 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1308/2628] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:45.468392 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aee442-efcf-4cbc-8b8d-3cea121659ff None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:45.478586 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aee442-efcf-4cbc-8b8d-3cea121659ff None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:45.483721 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb93ed6b-e980-47b5-9141-111743285345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:45.484462 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93ed6b-e980-47b5-9141-111743285345 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:45.484748 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93ed6b-e980-47b5-9141-111743285345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:45.485187 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93ed6b-e980-47b5-9141-111743285345 admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:45.485948 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb93ed6b-e980-47b5-9141-111743285345 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:20:45.486268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb93ed6b-e980-47b5-9141-111743285345 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:45.487873 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb93ed6b-e980-47b5-9141-111743285345 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:45.501066 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1321/2629] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:20:45.505740 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f59aa0-f31c-42a1-8e32-1ed2b2d5da5e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:45.516863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f59aa0-f31c-42a1-8e32-1ed2b2d5da5e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:45.521969 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b87e7-6126-491d-abf7-007f35423a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:45.522659 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b87e7-6126-491d-abf7-007f35423a93 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:45.522935 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b87e7-6126-491d-abf7-007f35423a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:45.523206 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b87e7-6126-491d-abf7-007f35423a93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:45.523903 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b87e7-6126-491d-abf7-007f35423a93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:45.525850 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b87e7-6126-491d-abf7-007f35423a93 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:45.555115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1309/2630] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:20:45 2022] POST /identity/v3/projects => generated 389 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:45.559439 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb93ed6b-e980-47b5-9141-111743285345 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:45.571019 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb93ed6b-e980-47b5-9141-111743285345 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:45.636296 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42f34f-36de-4aec-b5ff-4e7cd3ebe06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:45.636976 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42f34f-36de-4aec-b5ff-4e7cd3ebe06d admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:45.637263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42f34f-36de-4aec-b5ff-4e7cd3ebe06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:45.637541 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42f34f-36de-4aec-b5ff-4e7cd3ebe06d admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:45.638285 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42f34f-36de-4aec-b5ff-4e7cd3ebe06d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:45.640876 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42f34f-36de-4aec-b5ff-4e7cd3ebe06d admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:45.662579 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1322/2631] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:20:45 2022] POST /identity/v3/users => generated 295 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:45.666687 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b87e7-6126-491d-abf7-007f35423a93 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:45.677697 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b87e7-6126-491d-abf7-007f35423a93 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:45.685572 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a780466-4f9b-4e2a-8de2-05da5dfaea9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:45.685572 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a780466-4f9b-4e2a-8de2-05da5dfaea9d admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:45.685572 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a780466-4f9b-4e2a-8de2-05da5dfaea9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:45.685572 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a780466-4f9b-4e2a-8de2-05da5dfaea9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:45.687057 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a780466-4f9b-4e2a-8de2-05da5dfaea9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:45.687416 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a780466-4f9b-4e2a-8de2-05da5dfaea9d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:45.698939 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1310/2632] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:45.704694 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42f34f-36de-4aec-b5ff-4e7cd3ebe06d admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:45.715081 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42f34f-36de-4aec-b5ff-4e7cd3ebe06d admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:45.724668 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c61eb1-15a5-462c-a405-e6851d43f353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:45.725444 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c61eb1-15a5-462c-a405-e6851d43f353 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:45.725803 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c61eb1-15a5-462c-a405-e6851d43f353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:45.726171 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c61eb1-15a5-462c-a405-e6851d43f353 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:45.727040 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c61eb1-15a5-462c-a405-e6851d43f353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:45.728855 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c61eb1-15a5-462c-a405-e6851d43f353 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:45.743518 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1323/2633] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:45.745321 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a780466-4f9b-4e2a-8de2-05da5dfaea9d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:45.756290 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a780466-4f9b-4e2a-8de2-05da5dfaea9d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:45.762021 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c252f-b001-4bcf-b71d-90534da518ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:45.762790 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c252f-b001-4bcf-b71d-90534da518ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:45.763074 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c252f-b001-4bcf-b71d-90534da518ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:45.763349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c252f-b001-4bcf-b71d-90534da518ad admin admin] PATH_INFO: `/v3/projects/bf848c0546d34112a35d72ff5f7e7c68/users/8b9f7660cc4e4639bf251c14baf6b127/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:45.780609 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c252f-b001-4bcf-b71d-90534da518ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf848c0546d34112a35d72ff5f7e7c68, user_id=8b9f7660cc4e4639bf251c14baf6b127, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:45.783414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c252f-b001-4bcf-b71d-90534da518ad admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:45.800477 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1311/2634] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:45 2022] PUT /identity/v3/projects/bf848c0546d34112a35d72ff5f7e7c68/users/8b9f7660cc4e4639bf251c14baf6b127/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:20:45.813301 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a983e9-4663-4667-941e-c75c5aef2e52 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:45.813649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a983e9-4663-4667-941e-c75c5aef2e52 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:45.813950 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a983e9-4663-4667-941e-c75c5aef2e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:45.843651 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-83a983e9-4663-4667-941e-c75c5aef2e52 None None] MFA Rules not processed for user `8b9f7660cc4e4639bf251c14baf6b127`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:45.844893 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-83a983e9-4663-4667-941e-c75c5aef2e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:45.850421 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:45.932997 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1324/2635] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:45.939453 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c252f-b001-4bcf-b71d-90534da518ad admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:46.020026 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c252f-b001-4bcf-b71d-90534da518ad admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:46.025093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de24ff-35c7-495b-a487-f22a619ed6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:46.025840 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de24ff-35c7-495b-a487-f22a619ed6ff admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:46.026094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de24ff-35c7-495b-a487-f22a619ed6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:46.026374 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de24ff-35c7-495b-a487-f22a619ed6ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:46.027075 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de24ff-35c7-495b-a487-f22a619ed6ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:46.029110 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de24ff-35c7-495b-a487-f22a619ed6ff admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:46.058471 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1312/2636] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:20:45 2022] POST /identity/v3/projects => generated 389 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:46.062956 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a983e9-4663-4667-941e-c75c5aef2e52 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:46.074704 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a983e9-4663-4667-941e-c75c5aef2e52 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:46.141153 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d28a69-4435-4714-a6bc-9a3415caa69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:46.142044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d28a69-4435-4714-a6bc-9a3415caa69a admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:46.142338 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d28a69-4435-4714-a6bc-9a3415caa69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:46.142727 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d28a69-4435-4714-a6bc-9a3415caa69a admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:46.143541 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d28a69-4435-4714-a6bc-9a3415caa69a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:46.150298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d28a69-4435-4714-a6bc-9a3415caa69a admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:46.178421 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1325/2637] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:20:46 2022] POST /identity/v3/users => generated 301 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:46.179764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de24ff-35c7-495b-a487-f22a619ed6ff admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:46.193444 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de24ff-35c7-495b-a487-f22a619ed6ff admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:46.203525 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e766e9d0-bbd8-403d-9e96-f2bc912978a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:46.204335 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766e9d0-bbd8-403d-9e96-f2bc912978a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:46.204726 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766e9d0-bbd8-403d-9e96-f2bc912978a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:46.205133 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766e9d0-bbd8-403d-9e96-f2bc912978a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:46.206027 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e766e9d0-bbd8-403d-9e96-f2bc912978a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:46.207926 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e766e9d0-bbd8-403d-9e96-f2bc912978a8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:46.222105 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1313/2638] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:46.225764 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d28a69-4435-4714-a6bc-9a3415caa69a admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:46.236655 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d28a69-4435-4714-a6bc-9a3415caa69a admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:46.242090 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68bd31-cbdc-45a1-a733-e9be6bef5042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:46.243021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68bd31-cbdc-45a1-a733-e9be6bef5042 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:46.243304 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68bd31-cbdc-45a1-a733-e9be6bef5042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:46.243591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68bd31-cbdc-45a1-a733-e9be6bef5042 admin admin] PATH_INFO: `/v3/projects/4a572d3dbe304dfd865020a7e118874f/users/fa9c2c7999ca47b889bfe3163614a45c/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:46.262273 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68bd31-cbdc-45a1-a733-e9be6bef5042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a572d3dbe304dfd865020a7e118874f, user_id=fa9c2c7999ca47b889bfe3163614a45c, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:46.266392 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68bd31-cbdc-45a1-a733-e9be6bef5042 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:46.284173 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1326/2639] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:46 2022] PUT /identity/v3/projects/4a572d3dbe304dfd865020a7e118874f/users/fa9c2c7999ca47b889bfe3163614a45c/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:46.288105 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e766e9d0-bbd8-403d-9e96-f2bc912978a8 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:46.300763 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e766e9d0-bbd8-403d-9e96-f2bc912978a8 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:46.371796 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31da376d-bd7a-4af7-918f-086d9a5a0b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:46.372478 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da376d-bd7a-4af7-918f-086d9a5a0b02 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:46.372761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da376d-bd7a-4af7-918f-086d9a5a0b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:46.373037 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da376d-bd7a-4af7-918f-086d9a5a0b02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:46.374987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31da376d-bd7a-4af7-918f-086d9a5a0b02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:46.376621 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31da376d-bd7a-4af7-918f-086d9a5a0b02 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:46.382190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:46.388250 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1314/2640] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:46.392913 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68bd31-cbdc-45a1-a733-e9be6bef5042 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:46.403472 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68bd31-cbdc-45a1-a733-e9be6bef5042 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:46.408625 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67174c48-18d5-4402-b607-ebfd7dc9b9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:46.409518 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67174c48-18d5-4402-b607-ebfd7dc9b9f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:46.409812 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67174c48-18d5-4402-b607-ebfd7dc9b9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:46.410134 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67174c48-18d5-4402-b607-ebfd7dc9b9f7 admin admin] PATH_INFO: `/v3/projects/4a572d3dbe304dfd865020a7e118874f/users/fa9c2c7999ca47b889bfe3163614a45c/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:46.412237 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67174c48-18d5-4402-b607-ebfd7dc9b9f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a572d3dbe304dfd865020a7e118874f, user_id=fa9c2c7999ca47b889bfe3163614a45c, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:46.414961 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67174c48-18d5-4402-b607-ebfd7dc9b9f7 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:46.426215 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1327/2641] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:46 2022] PUT /identity/v3/projects/4a572d3dbe304dfd865020a7e118874f/users/fa9c2c7999ca47b889bfe3163614a45c/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:46.437175 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdc92a-539d-473e-af9b-3881928326b5 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:46.437662 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdc92a-539d-473e-af9b-3881928326b5 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:46.437784 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdc92a-539d-473e-af9b-3881928326b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:46.464619 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-21bdc92a-539d-473e-af9b-3881928326b5 None None] MFA Rules not processed for user `fa9c2c7999ca47b889bfe3163614a45c`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:46.465833 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-21bdc92a-539d-473e-af9b-3881928326b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:46.534933 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1315/2642] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:46 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:46.546221 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67174c48-18d5-4402-b607-ebfd7dc9b9f7 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:46.547965 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-67174c48-18d5-4402-b607-ebfd7dc9b9f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:46.605663 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67174c48-18d5-4402-b607-ebfd7dc9b9f7 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:46.610461 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1bdbcc-3857-4066-9528-6f2a57907c39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:46.611130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1bdbcc-3857-4066-9528-6f2a57907c39 service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:46.611419 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1bdbcc-3857-4066-9528-6f2a57907c39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:46.611700 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1bdbcc-3857-4066-9528-6f2a57907c39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:46.621715 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1bdbcc-3857-4066-9528-6f2a57907c39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:46.623574 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1bdbcc-3857-4066-9528-6f2a57907c39 service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:46.628787 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:46.634199 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1328/2643] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:20:46.772215 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bdc92a-539d-473e-af9b-3881928326b5 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:46.836636 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bdc92a-539d-473e-af9b-3881928326b5 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:46.841240 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a918c-9e2f-417a-9bd0-23ab695cd147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:46.842830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a918c-9e2f-417a-9bd0-23ab695cd147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:46.843116 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a918c-9e2f-417a-9bd0-23ab695cd147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:46.843395 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a918c-9e2f-417a-9bd0-23ab695cd147 admin admin] PATH_INFO: `/v3/users/8b9f7660cc4e4639bf251c14baf6b127` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:46.844607 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a918c-9e2f-417a-9bd0-23ab695cd147 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b9f7660cc4e4639bf251c14baf6b127)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:46.846577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a918c-9e2f-417a-9bd0-23ab695cd147 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:46.894558 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d40a918c-9e2f-417a-9bd0-23ab695cd147 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b9f7660cc4e4639bf251c14baf6b127'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:46.907741 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d40a918c-9e2f-417a-9bd0-23ab695cd147 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b9f7660cc4e4639bf251c14baf6b127'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:46.915326 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d40a918c-9e2f-417a-9bd0-23ab695cd147 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b9f7660cc4e4639bf251c14baf6b127'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:46.923064 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d40a918c-9e2f-417a-9bd0-23ab695cd147 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b9f7660cc4e4639bf251c14baf6b127'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:46.923790 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d40a918c-9e2f-417a-9bd0-23ab695cd147 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b9f7660cc4e4639bf251c14baf6b127'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:46.929980 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1316/2644] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:20:46 2022] DELETE /identity/v3/users/8b9f7660cc4e4639bf251c14baf6b127 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:46.934730 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83fa17-6c22-420b-97be-0e9163bef172 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:46.935022 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83fa17-6c22-420b-97be-0e9163bef172 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:46.935305 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83fa17-6c22-420b-97be-0e9163bef172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:46.953985 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-bb83fa17-6c22-420b-97be-0e9163bef172 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:46.955201 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-bb83fa17-6c22-420b-97be-0e9163bef172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:47.016109 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1329/2645] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:47.025874 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a918c-9e2f-417a-9bd0-23ab695cd147 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:47.027336 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d40a918c-9e2f-417a-9bd0-23ab695cd147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:47.034434 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:47.095197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a918c-9e2f-417a-9bd0-23ab695cd147 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:47.098873 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6192bb8-cf2d-414b-9805-1a1c6975b27f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:47.099530 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6192bb8-cf2d-414b-9805-1a1c6975b27f service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:47.099812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6192bb8-cf2d-414b-9805-1a1c6975b27f service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:47.100088 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6192bb8-cf2d-414b-9805-1a1c6975b27f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:47.110050 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6192bb8-cf2d-414b-9805-1a1c6975b27f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:47.111480 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6192bb8-cf2d-414b-9805-1a1c6975b27f service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:47.120921 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1317/2646] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:20:47.235148 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83fa17-6c22-420b-97be-0e9163bef172 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:47.236791 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-bb83fa17-6c22-420b-97be-0e9163bef172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:47.250314 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83fa17-6c22-420b-97be-0e9163bef172 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:47.255006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ac713-ce5d-450b-ae04-4b936f97c92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:47.255895 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ac713-ce5d-450b-ae04-4b936f97c92a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:47.256180 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ac713-ce5d-450b-ae04-4b936f97c92a admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:47.256461 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ac713-ce5d-450b-ae04-4b936f97c92a admin admin] PATH_INFO: `/v3/users/a76571e533064b1ba007b5d1a025c3cf` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:47.257681 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4ac713-ce5d-450b-ae04-4b936f97c92a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a76571e533064b1ba007b5d1a025c3cf)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:47.259956 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4ac713-ce5d-450b-ae04-4b936f97c92a admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:47.315392 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-0a4ac713-ce5d-450b-ae04-4b936f97c92a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a76571e533064b1ba007b5d1a025c3cf'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:47.323578 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-0a4ac713-ce5d-450b-ae04-4b936f97c92a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a76571e533064b1ba007b5d1a025c3cf'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:47.339499 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-0a4ac713-ce5d-450b-ae04-4b936f97c92a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a76571e533064b1ba007b5d1a025c3cf'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:47.340312 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-0a4ac713-ce5d-450b-ae04-4b936f97c92a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a76571e533064b1ba007b5d1a025c3cf'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:47.351164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-0a4ac713-ce5d-450b-ae04-4b936f97c92a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a76571e533064b1ba007b5d1a025c3cf'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:47.359816 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1330/2647] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:20:47 2022] DELETE /identity/v3/users/a76571e533064b1ba007b5d1a025c3cf => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:47.364164 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d03d3-9e34-47f4-bd1d-e0113dcf8aff None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:47.364640 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d03d3-9e34-47f4-bd1d-e0113dcf8aff None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:47.365103 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d03d3-9e34-47f4-bd1d-e0113dcf8aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:47.388920 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-865d03d3-9e34-47f4-bd1d-e0113dcf8aff None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:47.390560 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-865d03d3-9e34-47f4-bd1d-e0113dcf8aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:47.446101 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ac713-ce5d-450b-ae04-4b936f97c92a admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:47.448400 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-0a4ac713-ce5d-450b-ae04-4b936f97c92a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:47.468893 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1318/2648] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:20:47.479667 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d03d3-9e34-47f4-bd1d-e0113dcf8aff None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:47.481321 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-865d03d3-9e34-47f4-bd1d-e0113dcf8aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:47.556657 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d03d3-9e34-47f4-bd1d-e0113dcf8aff None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:47.563655 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff512ce-bc7e-4b4d-871c-d24f57c55e66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:47.565399 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff512ce-bc7e-4b4d-871c-d24f57c55e66 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:47.566131 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff512ce-bc7e-4b4d-871c-d24f57c55e66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:47.566705 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff512ce-bc7e-4b4d-871c-d24f57c55e66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:47.568649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ac713-ce5d-450b-ae04-4b936f97c92a admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:47.573802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19521ea6-1e66-46dd-aa10-fc79acb7d91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:47.574689 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19521ea6-1e66-46dd-aa10-fc79acb7d91b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:47.574973 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19521ea6-1e66-46dd-aa10-fc79acb7d91b admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:47.575258 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19521ea6-1e66-46dd-aa10-fc79acb7d91b admin admin] PATH_INFO: `/v3/projects/bf848c0546d34112a35d72ff5f7e7c68` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:47.577211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19521ea6-1e66-46dd-aa10-fc79acb7d91b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf848c0546d34112a35d72ff5f7e7c68)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:47.579032 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff512ce-bc7e-4b4d-871c-d24f57c55e66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:47.579702 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19521ea6-1e66-46dd-aa10-fc79acb7d91b admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:47.580917 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff512ce-bc7e-4b4d-871c-d24f57c55e66 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:47.598329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1319/2649] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:20:47.601693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:47.650486 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-19521ea6-1e66-46dd-aa10-fc79acb7d91b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf848c0546d34112a35d72ff5f7e7c68', 'request_id': 'req-19521ea6-1e66-46dd-aa10-fc79acb7d91b'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:47.660184 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-19521ea6-1e66-46dd-aa10-fc79acb7d91b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf848c0546d34112a35d72ff5f7e7c68', 'request_id': 'req-19521ea6-1e66-46dd-aa10-fc79acb7d91b'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:47.669337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-19521ea6-1e66-46dd-aa10-fc79acb7d91b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf848c0546d34112a35d72ff5f7e7c68', 'request_id': 'req-19521ea6-1e66-46dd-aa10-fc79acb7d91b'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:47.682436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-19521ea6-1e66-46dd-aa10-fc79acb7d91b admin admin] The token cache is being invalidate because project bf848c0546d34112a35d72ff5f7e7c68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:20:47.682730 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-19521ea6-1e66-46dd-aa10-fc79acb7d91b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:47.683547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-19521ea6-1e66-46dd-aa10-fc79acb7d91b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:47.685171 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1331/2650] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:20:47 2022] DELETE /identity/v3/projects/bf848c0546d34112a35d72ff5f7e7c68 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:47.691391 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff512ce-bc7e-4b4d-871c-d24f57c55e66 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:47.703071 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff512ce-bc7e-4b4d-871c-d24f57c55e66 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:47.761792 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:47.776662 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09386eea-4ca4-467a-804d-76697c72423d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:47.777489 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09386eea-4ca4-467a-804d-76697c72423d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:47.777772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09386eea-4ca4-467a-804d-76697c72423d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:47.778079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09386eea-4ca4-467a-804d-76697c72423d admin admin] PATH_INFO: `/v3/users/fa9c2c7999ca47b889bfe3163614a45c` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:47.780925 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09386eea-4ca4-467a-804d-76697c72423d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa9c2c7999ca47b889bfe3163614a45c)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:47.783039 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09386eea-4ca4-467a-804d-76697c72423d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:47.821197 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19521ea6-1e66-46dd-aa10-fc79acb7d91b admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:47.822800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-19521ea6-1e66-46dd-aa10-fc79acb7d91b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:47.843974 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19521ea6-1e66-46dd-aa10-fc79acb7d91b admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:47.853315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89daab-20d1-4e78-9614-144a6740f36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:47.855464 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89daab-20d1-4e78-9614-144a6740f36f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:47.855840 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89daab-20d1-4e78-9614-144a6740f36f admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:47.857267 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89daab-20d1-4e78-9614-144a6740f36f admin admin] PATH_INFO: `/v3/projects/765cab7190f642e891d2e7236edc3a8a` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:47.862446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a89daab-20d1-4e78-9614-144a6740f36f admin admin] RBAC: Authorizing `identity:delete_project(project_id=765cab7190f642e891d2e7236edc3a8a)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:47.865843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a89daab-20d1-4e78-9614-144a6740f36f admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:47.885805 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-09386eea-4ca4-467a-804d-76697c72423d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa9c2c7999ca47b889bfe3163614a45c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:47.912162 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-09386eea-4ca4-467a-804d-76697c72423d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa9c2c7999ca47b889bfe3163614a45c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:47.927967 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-09386eea-4ca4-467a-804d-76697c72423d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa9c2c7999ca47b889bfe3163614a45c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:47.939441 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-09386eea-4ca4-467a-804d-76697c72423d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa9c2c7999ca47b889bfe3163614a45c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:47.939685 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-0a89daab-20d1-4e78-9614-144a6740f36f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '765cab7190f642e891d2e7236edc3a8a', 'request_id': 'req-0a89daab-20d1-4e78-9614-144a6740f36f'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:47.940664 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-09386eea-4ca4-467a-804d-76697c72423d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa9c2c7999ca47b889bfe3163614a45c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:47.949229 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-0a89daab-20d1-4e78-9614-144a6740f36f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '765cab7190f642e891d2e7236edc3a8a', 'request_id': 'req-0a89daab-20d1-4e78-9614-144a6740f36f'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:47.950575 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1320/2651] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:20:47 2022] DELETE /identity/v3/users/fa9c2c7999ca47b889bfe3163614a45c => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:47.977338 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-0a89daab-20d1-4e78-9614-144a6740f36f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '765cab7190f642e891d2e7236edc3a8a', 'request_id': 'req-0a89daab-20d1-4e78-9614-144a6740f36f'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:47.998110 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-0a89daab-20d1-4e78-9614-144a6740f36f admin admin] The token cache is being invalidate because project 765cab7190f642e891d2e7236edc3a8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:20:47.998696 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-0a89daab-20d1-4e78-9614-144a6740f36f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:47.999922 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-0a89daab-20d1-4e78-9614-144a6740f36f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:48.002371 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1332/2652] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:20:47 2022] DELETE /identity/v3/projects/765cab7190f642e891d2e7236edc3a8a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:48.007779 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc6a8c-84ff-4821-a1f5-2e8eddd2b3f9 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:48.008070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc6a8c-84ff-4821-a1f5-2e8eddd2b3f9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:48.008370 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc6a8c-84ff-4821-a1f5-2e8eddd2b3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:48.033472 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-02bc6a8c-84ff-4821-a1f5-2e8eddd2b3f9 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:48.034801 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-02bc6a8c-84ff-4821-a1f5-2e8eddd2b3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:48.152273 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1321/2653] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:48.158912 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e838ec08-4fb3-4e62-b8fe-95146122e2cc None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:48.159547 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e838ec08-4fb3-4e62-b8fe-95146122e2cc None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:48.160769 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e838ec08-4fb3-4e62-b8fe-95146122e2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:48.190887 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-e838ec08-4fb3-4e62-b8fe-95146122e2cc None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:48.192649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-e838ec08-4fb3-4e62-b8fe-95146122e2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:48.206322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1333/2654] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:20:48.211340 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc6a8c-84ff-4821-a1f5-2e8eddd2b3f9 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:48.223748 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc6a8c-84ff-4821-a1f5-2e8eddd2b3f9 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:48.230544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78dd42a-af28-4b6d-9999-10578f3dde26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:48.232181 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78dd42a-af28-4b6d-9999-10578f3dde26 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:48.232181 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78dd42a-af28-4b6d-9999-10578f3dde26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:48.233010 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78dd42a-af28-4b6d-9999-10578f3dde26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:48.234266 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78dd42a-af28-4b6d-9999-10578f3dde26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:20:48.234643 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78dd42a-af28-4b6d-9999-10578f3dde26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:48.236496 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78dd42a-af28-4b6d-9999-10578f3dde26 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:48.254843 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1322/2655] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:20:48.260046 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e838ec08-4fb3-4e62-b8fe-95146122e2cc None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:48.271911 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e838ec08-4fb3-4e62-b8fe-95146122e2cc None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:48.277524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d88f6-e6d0-4811-b894-16d1727d4645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:48.278820 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d88f6-e6d0-4811-b894-16d1727d4645 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:48.280209 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d88f6-e6d0-4811-b894-16d1727d4645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:48.280769 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d88f6-e6d0-4811-b894-16d1727d4645 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:48.281737 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d88f6-e6d0-4811-b894-16d1727d4645 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:48.283992 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d88f6-e6d0-4811-b894-16d1727d4645 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:48.317470 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1334/2656] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:20:48 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:48.322018 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78dd42a-af28-4b6d-9999-10578f3dde26 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:48.328079 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d88f6-e6d0-4811-b894-16d1727d4645 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:48.332126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-f40d88f6-e6d0-4811-b894-16d1727d4645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:48.335300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78dd42a-af28-4b6d-9999-10578f3dde26 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:48.420280 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:48.429398 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585982f-5717-4157-926b-40604181b3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:48.430364 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585982f-5717-4157-926b-40604181b3fa admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:48.430831 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585982f-5717-4157-926b-40604181b3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:48.431265 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585982f-5717-4157-926b-40604181b3fa admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:48.432086 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585982f-5717-4157-926b-40604181b3fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:48.434456 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585982f-5717-4157-926b-40604181b3fa admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:48.436477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:48.444209 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d88f6-e6d0-4811-b894-16d1727d4645 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:48.450398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47ec0b-467d-4251-bcb2-f741fdd87fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:48.451939 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47ec0b-467d-4251-bcb2-f741fdd87fbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:48.452645 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47ec0b-467d-4251-bcb2-f741fdd87fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:48.452886 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47ec0b-467d-4251-bcb2-f741fdd87fbc admin admin] PATH_INFO: `/v3/projects/4a572d3dbe304dfd865020a7e118874f` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:48.454296 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a47ec0b-467d-4251-bcb2-f741fdd87fbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a572d3dbe304dfd865020a7e118874f)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:48.456589 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a47ec0b-467d-4251-bcb2-f741fdd87fbc admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:48.468870 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1323/2657] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:20:48 2022] POST /identity/v3/users => generated 286 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:48.473864 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585982f-5717-4157-926b-40604181b3fa admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:48.485543 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585982f-5717-4157-926b-40604181b3fa admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:48.491462 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5507b-fd38-422c-af0e-7439671fb8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:48.492151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5507b-fd38-422c-af0e-7439671fb8c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:48.492483 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5507b-fd38-422c-af0e-7439671fb8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:48.492810 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5507b-fd38-422c-af0e-7439671fb8c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:48.493915 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5507b-fd38-422c-af0e-7439671fb8c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:48.496154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5507b-fd38-422c-af0e-7439671fb8c8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:48.508636 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1324/2658] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:48.512974 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5507b-fd38-422c-af0e-7439671fb8c8 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:48.523885 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-3a47ec0b-467d-4251-bcb2-f741fdd87fbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a572d3dbe304dfd865020a7e118874f', 'request_id': 'req-3a47ec0b-467d-4251-bcb2-f741fdd87fbc'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:48.524110 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5507b-fd38-422c-af0e-7439671fb8c8 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:48.529603 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-3a47ec0b-467d-4251-bcb2-f741fdd87fbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a572d3dbe304dfd865020a7e118874f', 'request_id': 'req-3a47ec0b-467d-4251-bcb2-f741fdd87fbc'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:48.538593 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-3a47ec0b-467d-4251-bcb2-f741fdd87fbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a572d3dbe304dfd865020a7e118874f', 'request_id': 'req-3a47ec0b-467d-4251-bcb2-f741fdd87fbc'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:48.553177 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-3a47ec0b-467d-4251-bcb2-f741fdd87fbc admin admin] The token cache is being invalidate because project 4a572d3dbe304dfd865020a7e118874f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:20:48.553631 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-3a47ec0b-467d-4251-bcb2-f741fdd87fbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:48.554594 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-3a47ec0b-467d-4251-bcb2-f741fdd87fbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:48.556892 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1335/2659] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:20:48 2022] DELETE /identity/v3/projects/4a572d3dbe304dfd865020a7e118874f => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:48.562893 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706e001-fb9c-41c9-99f6-15053f472d94 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:48.563445 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706e001-fb9c-41c9-99f6-15053f472d94 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:48.563994 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706e001-fb9c-41c9-99f6-15053f472d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:48.586552 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-5706e001-fb9c-41c9-99f6-15053f472d94 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:48.587966 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-5706e001-fb9c-41c9-99f6-15053f472d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:48.592738 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd7c03-d62e-45ba-ae9f-9accb37e2465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:48.593455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd7c03-d62e-45ba-ae9f-9accb37e2465 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:48.593741 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd7c03-d62e-45ba-ae9f-9accb37e2465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:48.594053 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd7c03-d62e-45ba-ae9f-9accb37e2465 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:48.594790 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd7c03-d62e-45ba-ae9f-9accb37e2465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:48.596426 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd7c03-d62e-45ba-ae9f-9accb37e2465 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:48.597486 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1336/2660] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:48.607290 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd162a1d-5d1b-48e2-95e9-1225dc4cd674 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:48.607649 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd162a1d-5d1b-48e2-95e9-1225dc4cd674 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:48.607852 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd162a1d-5d1b-48e2-95e9-1225dc4cd674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:48.613002 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1325/2661] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:20:48.618006 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd7c03-d62e-45ba-ae9f-9accb37e2465 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:48.620371 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-dfdd7c03-d62e-45ba-ae9f-9accb37e2465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:48.630003 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-dd162a1d-5d1b-48e2-95e9-1225dc4cd674 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:48.631591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-dd162a1d-5d1b-48e2-95e9-1225dc4cd674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:48.636636 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd7c03-d62e-45ba-ae9f-9accb37e2465 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:48.645312 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1337/2662] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:48.647019 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd162a1d-5d1b-48e2-95e9-1225dc4cd674 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:48.647298 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee630b-bb33-45f1-b977-b4476505e03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:48.648226 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee630b-bb33-45f1-b977-b4476505e03e admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:48.648726 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee630b-bb33-45f1-b977-b4476505e03e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:48.648837 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee630b-bb33-45f1-b977-b4476505e03e admin admin] PATH_INFO: `/v3/projects/bd7ec755d96f4dee964a003d6c269c4e/users/080c7129a52e49c698cb2dc4afe14de7/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:48.658056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd162a1d-5d1b-48e2-95e9-1225dc4cd674 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:48.664412 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6305ad7-0d5c-4be0-8fe5-f5e83251b211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:48.665093 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6305ad7-0d5c-4be0-8fe5-f5e83251b211 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:48.665383 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6305ad7-0d5c-4be0-8fe5-f5e83251b211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:48.665661 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6305ad7-0d5c-4be0-8fe5-f5e83251b211 admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:48.666459 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6305ad7-0d5c-4be0-8fe5-f5e83251b211 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:20:48.666770 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6305ad7-0d5c-4be0-8fe5-f5e83251b211 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:48.668479 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6305ad7-0d5c-4be0-8fe5-f5e83251b211 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:48.670405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee630b-bb33-45f1-b977-b4476505e03e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd7ec755d96f4dee964a003d6c269c4e, user_id=080c7129a52e49c698cb2dc4afe14de7, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:48.673050 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee630b-bb33-45f1-b977-b4476505e03e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:48.683658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1338/2663] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:20:48.689157 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6305ad7-0d5c-4be0-8fe5-f5e83251b211 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:48.695206 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1326/2664] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:48 2022] PUT /identity/v3/projects/bd7ec755d96f4dee964a003d6c269c4e/users/080c7129a52e49c698cb2dc4afe14de7/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:20:48.700275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e9c7a-4528-4bf4-a520-ab19eca48bc0 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:48.700561 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e9c7a-4528-4bf4-a520-ab19eca48bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:48.700839 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e9c7a-4528-4bf4-a520-ab19eca48bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:48.708714 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6305ad7-0d5c-4be0-8fe5-f5e83251b211 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:48.723389 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-3e1e9c7a-4528-4bf4-a520-ab19eca48bc0 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:48.724851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3e1e9c7a-4528-4bf4-a520-ab19eca48bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:48.805429 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb34443-0633-4834-bdf4-67258f89f712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:48.806662 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb34443-0633-4834-bdf4-67258f89f712 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:48.806662 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb34443-0633-4834-bdf4-67258f89f712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:48.807266 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb34443-0633-4834-bdf4-67258f89f712 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:48.808418 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb34443-0633-4834-bdf4-67258f89f712 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:48.812102 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb34443-0633-4834-bdf4-67258f89f712 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:48.826083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1327/2665] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:48.837145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e9c7a-4528-4bf4-a520-ab19eca48bc0 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:48.838849 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3e1e9c7a-4528-4bf4-a520-ab19eca48bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:48.853399 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1339/2666] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:20:48 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:48.859230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb34443-0633-4834-bdf4-67258f89f712 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:48.913561 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:48.913955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:48.919228 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e9c7a-4528-4bf4-a520-ab19eca48bc0 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:48.934776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb34443-0633-4834-bdf4-67258f89f712 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:48.940592 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29986ad-f433-432f-af9b-b43446b72986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:48.941283 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29986ad-f433-432f-af9b-b43446b72986 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:48.941566 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29986ad-f433-432f-af9b-b43446b72986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:48.941863 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29986ad-f433-432f-af9b-b43446b72986 admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:48.944103 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29986ad-f433-432f-af9b-b43446b72986 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:48.944600 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29986ad-f433-432f-af9b-b43446b72986 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:48.965949 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1340/2667] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:20:48 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:20:48.970653 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29986ad-f433-432f-af9b-b43446b72986 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:48.980626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81e4b6-454a-4000-b301-860de2a816a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:48.981446 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81e4b6-454a-4000-b301-860de2a816a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:48.981738 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81e4b6-454a-4000-b301-860de2a816a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:48.982086 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81e4b6-454a-4000-b301-860de2a816a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:48.988094 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29986ad-f433-432f-af9b-b43446b72986 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:48.994591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5267ae-8b65-45cc-8b3a-8f6f543870fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:48.994878 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf81e4b6-454a-4000-b301-860de2a816a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:48.995273 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5267ae-8b65-45cc-8b3a-8f6f543870fc admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:48.995558 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5267ae-8b65-45cc-8b3a-8f6f543870fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:48.996187 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5267ae-8b65-45cc-8b3a-8f6f543870fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:48.996419 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf81e4b6-454a-4000-b301-860de2a816a9 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:48.996968 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5267ae-8b65-45cc-8b3a-8f6f543870fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:48.999449 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5267ae-8b65-45cc-8b3a-8f6f543870fc admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:49.010328 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1328/2668] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:20:49.012290 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1341/2669] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:20:49.017153 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81e4b6-454a-4000-b301-860de2a816a9 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:49.027498 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81e4b6-454a-4000-b301-860de2a816a9 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:49.034146 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8281c20-a757-4abb-970e-8589ff9f6045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:49.034861 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8281c20-a757-4abb-970e-8589ff9f6045 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:49.035145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8281c20-a757-4abb-970e-8589ff9f6045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:49.035429 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8281c20-a757-4abb-970e-8589ff9f6045 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:49.036175 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8281c20-a757-4abb-970e-8589ff9f6045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:49.038123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8281c20-a757-4abb-970e-8589ff9f6045 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:49.053720 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1329/2670] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:49.059132 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5267ae-8b65-45cc-8b3a-8f6f543870fc admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:49.071353 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5267ae-8b65-45cc-8b3a-8f6f543870fc admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:49.078402 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d321ba-552a-4412-baac-dba0cac64666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:49.079348 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d321ba-552a-4412-baac-dba0cac64666 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:49.079640 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d321ba-552a-4412-baac-dba0cac64666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:49.079924 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d321ba-552a-4412-baac-dba0cac64666 admin admin] PATH_INFO: `/v3/projects/8f6acdefc39341b083c2eda32d8f77bb/users/9715012b90dc44a8ae565d9c11fb73e6/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:49.099755 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d321ba-552a-4412-baac-dba0cac64666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f6acdefc39341b083c2eda32d8f77bb, user_id=9715012b90dc44a8ae565d9c11fb73e6, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:49.102863 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d321ba-552a-4412-baac-dba0cac64666 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:49.119766 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1342/2671] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:49 2022] PUT /identity/v3/projects/8f6acdefc39341b083c2eda32d8f77bb/users/9715012b90dc44a8ae565d9c11fb73e6/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:20:49.133424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9da5c-000b-46d3-a898-526f884888f6 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:49.133725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9da5c-000b-46d3-a898-526f884888f6 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:49.134411 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9da5c-000b-46d3-a898-526f884888f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:49.170493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-89a9da5c-000b-46d3-a898-526f884888f6 None None] MFA Rules not processed for user `9715012b90dc44a8ae565d9c11fb73e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:49.172084 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-89a9da5c-000b-46d3-a898-526f884888f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:49.276662 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1330/2672] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:49.291784 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d321ba-552a-4412-baac-dba0cac64666 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:49.293615 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-72d321ba-552a-4412-baac-dba0cac64666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:49.364425 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d321ba-552a-4412-baac-dba0cac64666 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:49.368653 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bee0c8-4410-4953-ad04-10b0bfc903d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:49.369380 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bee0c8-4410-4953-ad04-10b0bfc903d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:49.369685 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bee0c8-4410-4953-ad04-10b0bfc903d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:49.370049 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bee0c8-4410-4953-ad04-10b0bfc903d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:49.380270 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bee0c8-4410-4953-ad04-10b0bfc903d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:49.381831 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bee0c8-4410-4953-ad04-10b0bfc903d3 service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:49.393833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1343/2673] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:20:49.417070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9da5c-000b-46d3-a898-526f884888f6 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:49.516132 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9da5c-000b-46d3-a898-526f884888f6 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:49.521568 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b4807-45f1-47f9-a156-6df6ef580dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:49.522478 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b4807-45f1-47f9-a156-6df6ef580dfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:49.522763 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b4807-45f1-47f9-a156-6df6ef580dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:49.523038 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b4807-45f1-47f9-a156-6df6ef580dfc admin admin] PATH_INFO: `/v3/users/9715012b90dc44a8ae565d9c11fb73e6` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:49.524254 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b4807-45f1-47f9-a156-6df6ef580dfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9715012b90dc44a8ae565d9c11fb73e6)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:49.526342 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b4807-45f1-47f9-a156-6df6ef580dfc admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:49.575423 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:49.601545 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8b1b4807-45f1-47f9-a156-6df6ef580dfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9715012b90dc44a8ae565d9c11fb73e6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:49.622361 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8b1b4807-45f1-47f9-a156-6df6ef580dfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9715012b90dc44a8ae565d9c11fb73e6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:49.648380 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8b1b4807-45f1-47f9-a156-6df6ef580dfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9715012b90dc44a8ae565d9c11fb73e6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:49.668029 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8b1b4807-45f1-47f9-a156-6df6ef580dfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9715012b90dc44a8ae565d9c11fb73e6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:49.668803 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8b1b4807-45f1-47f9-a156-6df6ef580dfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9715012b90dc44a8ae565d9c11fb73e6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:49.679405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1331/2674] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:20:49 2022] DELETE /identity/v3/users/9715012b90dc44a8ae565d9c11fb73e6 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:49.684433 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0582be-4956-4158-8999-d62c477e5836 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:49.685206 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0582be-4956-4158-8999-d62c477e5836 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:49.685206 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0582be-4956-4158-8999-d62c477e5836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:49.711352 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-4c0582be-4956-4158-8999-d62c477e5836 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:49.713522 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-4c0582be-4956-4158-8999-d62c477e5836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:49.828239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b4807-45f1-47f9-a156-6df6ef580dfc admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:49.828239 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8b1b4807-45f1-47f9-a156-6df6ef580dfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:49.831062 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1344/2675] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:49.917283 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b4807-45f1-47f9-a156-6df6ef580dfc admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:49.926586 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e065b-6ee6-4086-911b-a707134fd096 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:49.928827 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e065b-6ee6-4086-911b-a707134fd096 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:49.929223 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e065b-6ee6-4086-911b-a707134fd096 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:49.930902 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e065b-6ee6-4086-911b-a707134fd096 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:49.949286 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e065b-6ee6-4086-911b-a707134fd096 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:49.953857 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e065b-6ee6-4086-911b-a707134fd096 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:49.972654 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1332/2676] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:20:50.375192 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0582be-4956-4158-8999-d62c477e5836 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:50.377739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-4c0582be-4956-4158-8999-d62c477e5836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:50.398595 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0582be-4956-4158-8999-d62c477e5836 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:50.408559 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca3c51-b770-44fd-bd00-6497a6116b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:50.409414 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca3c51-b770-44fd-bd00-6497a6116b03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:50.410152 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca3c51-b770-44fd-bd00-6497a6116b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:50.410152 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca3c51-b770-44fd-bd00-6497a6116b03 admin admin] PATH_INFO: `/v3/projects/8f6acdefc39341b083c2eda32d8f77bb` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:50.411325 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca3c51-b770-44fd-bd00-6497a6116b03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f6acdefc39341b083c2eda32d8f77bb)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:50.413430 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca3c51-b770-44fd-bd00-6497a6116b03 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:50.513573 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:50.529763 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-baca3c51-b770-44fd-bd00-6497a6116b03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f6acdefc39341b083c2eda32d8f77bb', 'request_id': 'req-baca3c51-b770-44fd-bd00-6497a6116b03'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:50.542008 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-baca3c51-b770-44fd-bd00-6497a6116b03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f6acdefc39341b083c2eda32d8f77bb', 'request_id': 'req-baca3c51-b770-44fd-bd00-6497a6116b03'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:50.555921 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-baca3c51-b770-44fd-bd00-6497a6116b03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f6acdefc39341b083c2eda32d8f77bb', 'request_id': 'req-baca3c51-b770-44fd-bd00-6497a6116b03'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:50.566095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491974c1-4524-4218-8688-3e6c6cf6ee47 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:50.566733 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491974c1-4524-4218-8688-3e6c6cf6ee47 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:50.567398 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491974c1-4524-4218-8688-3e6c6cf6ee47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:50.582455 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-baca3c51-b770-44fd-bd00-6497a6116b03 admin admin] The token cache is being invalidate because project 8f6acdefc39341b083c2eda32d8f77bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:20:50.583029 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-baca3c51-b770-44fd-bd00-6497a6116b03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:50.584906 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-baca3c51-b770-44fd-bd00-6497a6116b03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:50.591022 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-491974c1-4524-4218-8688-3e6c6cf6ee47 None None] MFA Rules not processed for user `2c2e0e19880f4b048faf034dd38c7365`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:50.591022 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-491974c1-4524-4218-8688-3e6c6cf6ee47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:50.591363 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1345/2677] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:20:50 2022] DELETE /identity/v3/projects/8f6acdefc39341b083c2eda32d8f77bb => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:50.596372 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea13f7-f758-4364-82a4-ebecca6c6a08 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:50.596980 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea13f7-f758-4364-82a4-ebecca6c6a08 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:50.598074 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea13f7-f758-4364-82a4-ebecca6c6a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:50.633404 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-5fea13f7-f758-4364-82a4-ebecca6c6a08 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:50.635748 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-5fea13f7-f758-4364-82a4-ebecca6c6a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:50.674443 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1333/2678] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 16:20:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:50.690860 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491974c1-4524-4218-8688-3e6c6cf6ee47 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:50.692723 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-491974c1-4524-4218-8688-3e6c6cf6ee47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:50.738675 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1346/2679] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:50.753077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e20de5-51c7-4927-be40-3a686088e636 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:50.754219 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e20de5-51c7-4927-be40-3a686088e636 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:50.754658 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e20de5-51c7-4927-be40-3a686088e636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:50.792338 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-a7e20de5-51c7-4927-be40-3a686088e636 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:50.798238 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-a7e20de5-51c7-4927-be40-3a686088e636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:50.799544 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:50.821154 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1347/2680] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:50.824173 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e20de5-51c7-4927-be40-3a686088e636 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:50.828049 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491974c1-4524-4218-8688-3e6c6cf6ee47 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:50.834348 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7bba17-e453-4d3a-9a9f-ba30437d2780 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0082e37ce54f49f7837601349508c89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:50.835101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7bba17-e453-4d3a-9a9f-ba30437d2780 service swift] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:50.835328 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e20de5-51c7-4927-be40-3a686088e636 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:50.835457 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7bba17-e453-4d3a-9a9f-ba30437d2780 service swift] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:50.835840 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7bba17-e453-4d3a-9a9f-ba30437d2780 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:50.842203 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3c64d-72cf-4458-81f9-28c173df8244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:50.845060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3c64d-72cf-4458-81f9-28c173df8244 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:50.845574 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3c64d-72cf-4458-81f9-28c173df8244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:50.846047 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3c64d-72cf-4458-81f9-28c173df8244 admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:50.847340 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a3c64d-72cf-4458-81f9-28c173df8244 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:20:50.847661 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a3c64d-72cf-4458-81f9-28c173df8244 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:50.847775 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7bba17-e453-4d3a-9a9f-ba30437d2780 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:50.849379 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a3c64d-72cf-4458-81f9-28c173df8244 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:50.849508 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7bba17-e453-4d3a-9a9f-ba30437d2780 service swift] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:50.865984 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1348/2681] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:20:50.866199 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1334/2682] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 16:20:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 16:20:50.877475 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3c64d-72cf-4458-81f9-28c173df8244 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:50.901477 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3c64d-72cf-4458-81f9-28c173df8244 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:50.917412 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774dde1-812a-456d-9330-1a39d0277609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:50.919012 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774dde1-812a-456d-9330-1a39d0277609 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:50.919304 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774dde1-812a-456d-9330-1a39d0277609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:50.919588 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774dde1-812a-456d-9330-1a39d0277609 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:50.920398 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774dde1-812a-456d-9330-1a39d0277609 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:50.922585 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774dde1-812a-456d-9330-1a39d0277609 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:50.997097 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1349/2683] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:20:50 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:51.001665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7bba17-e453-4d3a-9a9f-ba30437d2780 service swift] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:51.031505 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7bba17-e453-4d3a-9a9f-ba30437d2780 service swift] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:51.166306 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3196b-a025-4846-89cd-7aa148652642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:51.166990 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3196b-a025-4846-89cd-7aa148652642 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:51.167274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3196b-a025-4846-89cd-7aa148652642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:51.167547 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3196b-a025-4846-89cd-7aa148652642 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:51.168257 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3196b-a025-4846-89cd-7aa148652642 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:51.170903 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3196b-a025-4846-89cd-7aa148652642 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:51.200280 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1335/2684] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:20:51 2022] POST /identity/v3/users => generated 287 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:51.211465 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774dde1-812a-456d-9330-1a39d0277609 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:51.257360 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774dde1-812a-456d-9330-1a39d0277609 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:51.284003 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7af20-c4f6-4550-8f1e-27d9241ca4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:51.290821 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7af20-c4f6-4550-8f1e-27d9241ca4f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:51.291282 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7af20-c4f6-4550-8f1e-27d9241ca4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:51.291777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7af20-c4f6-4550-8f1e-27d9241ca4f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:51.298064 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa7af20-c4f6-4550-8f1e-27d9241ca4f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:51.300374 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa7af20-c4f6-4550-8f1e-27d9241ca4f7 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:51.350311 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1350/2685] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:51.354885 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3196b-a025-4846-89cd-7aa148652642 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:51.372577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3196b-a025-4846-89cd-7aa148652642 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:51.386266 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c89a4-1ebf-4f44-bb63-bfad3719b892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:51.387208 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c89a4-1ebf-4f44-bb63-bfad3719b892 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:51.391242 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c89a4-1ebf-4f44-bb63-bfad3719b892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:51.395431 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c89a4-1ebf-4f44-bb63-bfad3719b892 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:51.396390 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79c89a4-1ebf-4f44-bb63-bfad3719b892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:51.401843 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79c89a4-1ebf-4f44-bb63-bfad3719b892 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:51.428270 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1336/2686] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:51.429813 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7af20-c4f6-4550-8f1e-27d9241ca4f7 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:51.450888 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7af20-c4f6-4550-8f1e-27d9241ca4f7 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:51.466298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17ed86b-e14f-4c9a-ba67-a85f9e7ea7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:51.468363 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ed86b-e14f-4c9a-ba67-a85f9e7ea7b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:51.468640 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ed86b-e14f-4c9a-ba67-a85f9e7ea7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:51.468923 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ed86b-e14f-4c9a-ba67-a85f9e7ea7b7 admin admin] PATH_INFO: `/v3/projects/6619fc9a617f4bc1bdf5d2870d3ed8b0/users/9d72c500f1254e9d88bf32f07572adb9/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:51.519845 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17ed86b-e14f-4c9a-ba67-a85f9e7ea7b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6619fc9a617f4bc1bdf5d2870d3ed8b0, user_id=9d72c500f1254e9d88bf32f07572adb9, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:51.522662 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17ed86b-e14f-4c9a-ba67-a85f9e7ea7b7 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:51.569847 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1351/2687] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:51 2022] PUT /identity/v3/projects/6619fc9a617f4bc1bdf5d2870d3ed8b0/users/9d72c500f1254e9d88bf32f07572adb9/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:51.582178 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901aad09-2c84-4c6c-9d60-2eb7de52511a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:51.582802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901aad09-2c84-4c6c-9d60-2eb7de52511a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:51.582802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901aad09-2c84-4c6c-9d60-2eb7de52511a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:51.676638 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-901aad09-2c84-4c6c-9d60-2eb7de52511a None None] MFA Rules not processed for user `9d72c500f1254e9d88bf32f07572adb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:51.677881 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-901aad09-2c84-4c6c-9d60-2eb7de52511a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:51.856186 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1337/2688] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:51.861349 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17ed86b-e14f-4c9a-ba67-a85f9e7ea7b7 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:51.933315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:51.955605 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17ed86b-e14f-4c9a-ba67-a85f9e7ea7b7 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:51.964228 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca27029-5854-4bfb-92c8-180dad623786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:51.965367 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca27029-5854-4bfb-92c8-180dad623786 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:51.965827 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca27029-5854-4bfb-92c8-180dad623786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:51.966289 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca27029-5854-4bfb-92c8-180dad623786 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:51.967223 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca27029-5854-4bfb-92c8-180dad623786 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:51.969750 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca27029-5854-4bfb-92c8-180dad623786 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:52.017959 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1352/2689] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:20:51 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:52.019743 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901aad09-2c84-4c6c-9d60-2eb7de52511a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:52.033305 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901aad09-2c84-4c6c-9d60-2eb7de52511a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:52.112826 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:52.127550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1169f58-5c1d-4852-966e-dfd9c7cf87ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:52.127550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1169f58-5c1d-4852-966e-dfd9c7cf87ba admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:52.127550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1169f58-5c1d-4852-966e-dfd9c7cf87ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:52.127550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1169f58-5c1d-4852-966e-dfd9c7cf87ba admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:52.128538 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1169f58-5c1d-4852-966e-dfd9c7cf87ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:52.131527 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1169f58-5c1d-4852-966e-dfd9c7cf87ba admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:52.166439 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1338/2690] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:20:52 2022] POST /identity/v3/users => generated 293 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:52.171028 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca27029-5854-4bfb-92c8-180dad623786 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:52.187499 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca27029-5854-4bfb-92c8-180dad623786 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:52.193944 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4ef34-8719-442f-a710-a992e199b4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:52.194697 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4ef34-8719-442f-a710-a992e199b4ab admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:52.194985 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4ef34-8719-442f-a710-a992e199b4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:52.195274 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4ef34-8719-442f-a710-a992e199b4ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:52.196063 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4ef34-8719-442f-a710-a992e199b4ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:52.198427 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4ef34-8719-442f-a710-a992e199b4ab admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:52.217063 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1353/2691] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:20:52.221172 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1169f58-5c1d-4852-966e-dfd9c7cf87ba admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:52.239098 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1169f58-5c1d-4852-966e-dfd9c7cf87ba admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:52.252547 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110b98c-28af-4a62-bbf5-eb18cd9d9f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:52.253424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110b98c-28af-4a62-bbf5-eb18cd9d9f97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:52.253882 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110b98c-28af-4a62-bbf5-eb18cd9d9f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:52.254228 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110b98c-28af-4a62-bbf5-eb18cd9d9f97 admin admin] PATH_INFO: `/v3/projects/d7b7d463bbc34a06a48f40d4af548125/users/6a7eb6c39b754cd789c1e844422973af/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:52.278813 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110b98c-28af-4a62-bbf5-eb18cd9d9f97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7b7d463bbc34a06a48f40d4af548125, user_id=6a7eb6c39b754cd789c1e844422973af, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:52.284371 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110b98c-28af-4a62-bbf5-eb18cd9d9f97 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:52.317344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1339/2692] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:52 2022] PUT /identity/v3/projects/d7b7d463bbc34a06a48f40d4af548125/users/6a7eb6c39b754cd789c1e844422973af/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:52.320638 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4ef34-8719-442f-a710-a992e199b4ab admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:52.337076 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4ef34-8719-442f-a710-a992e199b4ab admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:52.493298 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6530326b-cf86-4839-b99a-78ba694c97ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:52.494198 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530326b-cf86-4839-b99a-78ba694c97ed admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:52.494587 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530326b-cf86-4839-b99a-78ba694c97ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:52.494965 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530326b-cf86-4839-b99a-78ba694c97ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:52.495831 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6530326b-cf86-4839-b99a-78ba694c97ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:52.498607 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6530326b-cf86-4839-b99a-78ba694c97ed admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:52.525720 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1354/2693] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:52.531525 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110b98c-28af-4a62-bbf5-eb18cd9d9f97 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:52.552529 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110b98c-28af-4a62-bbf5-eb18cd9d9f97 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:52.559979 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a5a3d-5bf9-40db-86e3-5c4da5de46b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:52.562123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a5a3d-5bf9-40db-86e3-5c4da5de46b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:52.562420 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a5a3d-5bf9-40db-86e3-5c4da5de46b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:52.562767 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a5a3d-5bf9-40db-86e3-5c4da5de46b2 admin admin] PATH_INFO: `/v3/projects/d7b7d463bbc34a06a48f40d4af548125/users/6a7eb6c39b754cd789c1e844422973af/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:52.565054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375a5a3d-5bf9-40db-86e3-5c4da5de46b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7b7d463bbc34a06a48f40d4af548125, user_id=6a7eb6c39b754cd789c1e844422973af, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:52.570448 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375a5a3d-5bf9-40db-86e3-5c4da5de46b2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:52.599437 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1340/2694] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:52 2022] PUT /identity/v3/projects/d7b7d463bbc34a06a48f40d4af548125/users/6a7eb6c39b754cd789c1e844422973af/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:52.614401 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcad68-0cb2-4107-bebd-be3dace10399 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:52.615220 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcad68-0cb2-4107-bebd-be3dace10399 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:52.615989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcad68-0cb2-4107-bebd-be3dace10399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:52.693877 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-9adcad68-0cb2-4107-bebd-be3dace10399 None None] MFA Rules not processed for user `6a7eb6c39b754cd789c1e844422973af`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:52.695140 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-9adcad68-0cb2-4107-bebd-be3dace10399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:52.877768 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1355/2695] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:52 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:52.888837 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a5a3d-5bf9-40db-86e3-5c4da5de46b2 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:52.890990 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-375a5a3d-5bf9-40db-86e3-5c4da5de46b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:53.038363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a5a3d-5bf9-40db-86e3-5c4da5de46b2 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:53.043818 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412603a-e137-4b7b-b48e-da5ebccaae09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:53.047268 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412603a-e137-4b7b-b48e-da5ebccaae09 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:53.047727 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412603a-e137-4b7b-b48e-da5ebccaae09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:53.048189 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412603a-e137-4b7b-b48e-da5ebccaae09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:53.143616 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412603a-e137-4b7b-b48e-da5ebccaae09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:53.145476 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412603a-e137-4b7b-b48e-da5ebccaae09 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:53.157670 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1341/2696] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:20:55.695480 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320c5f6-c473-4a9c-9302-3074c6d5c002 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:55.696038 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320c5f6-c473-4a9c-9302-3074c6d5c002 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:55.696038 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320c5f6-c473-4a9c-9302-3074c6d5c002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:55.744318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-e320c5f6-c473-4a9c-9302-3074c6d5c002 None None] MFA Rules not processed for user `080c7129a52e49c698cb2dc4afe14de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:55.745380 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-e320c5f6-c473-4a9c-9302-3074c6d5c002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:55.846131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:55.935846 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1356/2697] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:55.937064 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412603a-e137-4b7b-b48e-da5ebccaae09 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:55.938682 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4412603a-e137-4b7b-b48e-da5ebccaae09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:56.045566 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:56.051728 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412603a-e137-4b7b-b48e-da5ebccaae09 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:56.057289 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb2c85-85d1-4e90-9591-7d98682530e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:56.058532 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb2c85-85d1-4e90-9591-7d98682530e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:56.058906 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb2c85-85d1-4e90-9591-7d98682530e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:56.059333 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb2c85-85d1-4e90-9591-7d98682530e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:56.061164 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb2c85-85d1-4e90-9591-7d98682530e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:56.064008 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb2c85-85d1-4e90-9591-7d98682530e4 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:56.098354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1342/2698] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:20:55 2022] POST /identity/v3/projects => generated 371 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:56.102390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320c5f6-c473-4a9c-9302-3074c6d5c002 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:56.115722 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320c5f6-c473-4a9c-9302-3074c6d5c002 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:56.214035 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17485f-b6e3-49c5-80fa-63880a88d49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:56.214035 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17485f-b6e3-49c5-80fa-63880a88d49d admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:56.214035 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17485f-b6e3-49c5-80fa-63880a88d49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:56.214035 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17485f-b6e3-49c5-80fa-63880a88d49d admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:56.214478 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf17485f-b6e3-49c5-80fa-63880a88d49d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:56.216571 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf17485f-b6e3-49c5-80fa-63880a88d49d admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:56.256469 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1357/2699] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:20:56 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:20:56.268485 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb2c85-85d1-4e90-9591-7d98682530e4 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:56.293059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb2c85-85d1-4e90-9591-7d98682530e4 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:56.299707 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1d42e-6836-40ed-8917-a2b24a706d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:56.300491 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1d42e-6836-40ed-8917-a2b24a706d12 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:56.300865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1d42e-6836-40ed-8917-a2b24a706d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:56.301083 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1d42e-6836-40ed-8917-a2b24a706d12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:56.301825 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce1d42e-6836-40ed-8917-a2b24a706d12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:56.302106 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17485f-b6e3-49c5-80fa-63880a88d49d admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:56.304007 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce1d42e-6836-40ed-8917-a2b24a706d12 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:56.327056 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17485f-b6e3-49c5-80fa-63880a88d49d admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:56.332444 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ea80f-830b-4f61-a290-e7bf13f27829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:56.333664 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ea80f-830b-4f61-a290-e7bf13f27829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:56.334001 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ea80f-830b-4f61-a290-e7bf13f27829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:56.334289 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ea80f-830b-4f61-a290-e7bf13f27829 admin admin] PATH_INFO: `/v3/users/9d72c500f1254e9d88bf32f07572adb9` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:56.337430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1343/2700] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:56.338414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1d42e-6836-40ed-8917-a2b24a706d12 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:56.338728 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ea80f-830b-4f61-a290-e7bf13f27829 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d72c500f1254e9d88bf32f07572adb9)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:56.342982 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ea80f-830b-4f61-a290-e7bf13f27829 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:56.351920 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1d42e-6836-40ed-8917-a2b24a706d12 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:56.358748 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce4e15-119a-4e43-89c4-6ccd057caf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:56.359664 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce4e15-119a-4e43-89c4-6ccd057caf26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:56.359948 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce4e15-119a-4e43-89c4-6ccd057caf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:56.360296 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce4e15-119a-4e43-89c4-6ccd057caf26 admin admin] PATH_INFO: `/v3/projects/1ddadfa11c1c4dbd8b2dacc6ab97f722/users/2eeb3db00b9d4c3ba667bd25abe72b27/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:56.381865 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce4e15-119a-4e43-89c4-6ccd057caf26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ddadfa11c1c4dbd8b2dacc6ab97f722, user_id=2eeb3db00b9d4c3ba667bd25abe72b27, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:56.385314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce4e15-119a-4e43-89c4-6ccd057caf26 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:56.411124 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1344/2701] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:56 2022] PUT /identity/v3/projects/1ddadfa11c1c4dbd8b2dacc6ab97f722/users/2eeb3db00b9d4c3ba667bd25abe72b27/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:56.425048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce4e15-119a-4e43-89c4-6ccd057caf26 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:56.453214 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-105ea80f-830b-4f61-a290-e7bf13f27829 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d72c500f1254e9d88bf32f07572adb9'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:56.462044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-105ea80f-830b-4f61-a290-e7bf13f27829 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d72c500f1254e9d88bf32f07572adb9'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:56.490693 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-105ea80f-830b-4f61-a290-e7bf13f27829 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d72c500f1254e9d88bf32f07572adb9'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:56.491417 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-105ea80f-830b-4f61-a290-e7bf13f27829 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d72c500f1254e9d88bf32f07572adb9'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:56.505444 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-105ea80f-830b-4f61-a290-e7bf13f27829 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d72c500f1254e9d88bf32f07572adb9'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:56.522531 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1358/2702] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:20:56 2022] DELETE /identity/v3/users/9d72c500f1254e9d88bf32f07572adb9 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:20:56.523973 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce4e15-119a-4e43-89c4-6ccd057caf26 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:56.527008 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff727f0-a706-499f-99dd-1b509f7e3aa0 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:56.527401 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff727f0-a706-499f-99dd-1b509f7e3aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:56.527776 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff727f0-a706-499f-99dd-1b509f7e3aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:56.554883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-5ff727f0-a706-499f-99dd-1b509f7e3aa0 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:56.557115 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-5ff727f0-a706-499f-99dd-1b509f7e3aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:56.614265 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:56.619502 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c7e888-5ab5-4bc5-b2c9-f10ae2a509c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:56.620477 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7e888-5ab5-4bc5-b2c9-f10ae2a509c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:56.620965 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7e888-5ab5-4bc5-b2c9-f10ae2a509c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:56.621400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7e888-5ab5-4bc5-b2c9-f10ae2a509c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:56.622386 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c7e888-5ab5-4bc5-b2c9-f10ae2a509c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:56.624800 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c7e888-5ab5-4bc5-b2c9-f10ae2a509c5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:56.649986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1345/2703] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:20:56.651902 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c7e888-5ab5-4bc5-b2c9-f10ae2a509c5 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:56.653894 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-30c7e888-5ab5-4bc5-b2c9-f10ae2a509c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:56.665900 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1359/2704] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:56.674867 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c7e888-5ab5-4bc5-b2c9-f10ae2a509c5 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:56.676937 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff727f0-a706-499f-99dd-1b509f7e3aa0 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:56.681372 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-5ff727f0-a706-499f-99dd-1b509f7e3aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:56.687353 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13823b-ff21-4b5f-84e3-5b977173e367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:56.688275 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13823b-ff21-4b5f-84e3-5b977173e367 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:56.688561 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13823b-ff21-4b5f-84e3-5b977173e367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:56.688837 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13823b-ff21-4b5f-84e3-5b977173e367 admin admin] PATH_INFO: `/v3/projects/1ddadfa11c1c4dbd8b2dacc6ab97f722/users/2eeb3db00b9d4c3ba667bd25abe72b27/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:56.691150 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13823b-ff21-4b5f-84e3-5b977173e367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ddadfa11c1c4dbd8b2dacc6ab97f722, user_id=2eeb3db00b9d4c3ba667bd25abe72b27, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:56.694801 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13823b-ff21-4b5f-84e3-5b977173e367 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:56.710817 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1346/2705] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:20:56 2022] PUT /identity/v3/projects/1ddadfa11c1c4dbd8b2dacc6ab97f722/users/2eeb3db00b9d4c3ba667bd25abe72b27/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:20:56.776565 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff727f0-a706-499f-99dd-1b509f7e3aa0 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:56.888316 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f6d37-6bf8-45ed-8908-794e1330a84d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:56.916752 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f6d37-6bf8-45ed-8908-794e1330a84d service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:56.917085 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f6d37-6bf8-45ed-8908-794e1330a84d service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:56.917326 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f6d37-6bf8-45ed-8908-794e1330a84d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:57.058846 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666f6d37-6bf8-45ed-8908-794e1330a84d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:57.061042 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666f6d37-6bf8-45ed-8908-794e1330a84d service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:57.074697 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1360/2706] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:20:57.883019 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13823b-ff21-4b5f-84e3-5b977173e367 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:57.885156 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7e13823b-ff21-4b5f-84e3-5b977173e367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:57.921065 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13823b-ff21-4b5f-84e3-5b977173e367 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:57.938547 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f5c3c-ce07-4fed-bf63-6dc23da1d42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:57.939766 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f5c3c-ce07-4fed-bf63-6dc23da1d42f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:57.942820 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f5c3c-ce07-4fed-bf63-6dc23da1d42f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:57.943125 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f5c3c-ce07-4fed-bf63-6dc23da1d42f admin admin] PATH_INFO: `/v3/projects/6619fc9a617f4bc1bdf5d2870d3ed8b0` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:57.944951 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753f5c3c-ce07-4fed-bf63-6dc23da1d42f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6619fc9a617f4bc1bdf5d2870d3ed8b0)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:57.947037 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753f5c3c-ce07-4fed-bf63-6dc23da1d42f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:58.004157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:58.033439 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-753f5c3c-ce07-4fed-bf63-6dc23da1d42f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6619fc9a617f4bc1bdf5d2870d3ed8b0', 'request_id': 'req-753f5c3c-ce07-4fed-bf63-6dc23da1d42f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:58.040546 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-753f5c3c-ce07-4fed-bf63-6dc23da1d42f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6619fc9a617f4bc1bdf5d2870d3ed8b0', 'request_id': 'req-753f5c3c-ce07-4fed-bf63-6dc23da1d42f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:58.052702 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-753f5c3c-ce07-4fed-bf63-6dc23da1d42f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6619fc9a617f4bc1bdf5d2870d3ed8b0', 'request_id': 'req-753f5c3c-ce07-4fed-bf63-6dc23da1d42f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:58.073442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-753f5c3c-ce07-4fed-bf63-6dc23da1d42f admin admin] The token cache is being invalidate because project 6619fc9a617f4bc1bdf5d2870d3ed8b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:20:58.074711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-753f5c3c-ce07-4fed-bf63-6dc23da1d42f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:58.076055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-753f5c3c-ce07-4fed-bf63-6dc23da1d42f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:58.081812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1347/2707] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:20:57 2022] DELETE /identity/v3/projects/6619fc9a617f4bc1bdf5d2870d3ed8b0 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:58.087837 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f6d37-6bf8-45ed-8908-794e1330a84d service neutron] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:58.132160 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f6d37-6bf8-45ed-8908-794e1330a84d service neutron] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:58.247313 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:58.271021 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84d891-a64f-4e73-940f-26d5e16ee477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:58.272073 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84d891-a64f-4e73-940f-26d5e16ee477 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:58.272374 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84d891-a64f-4e73-940f-26d5e16ee477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:58.272667 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84d891-a64f-4e73-940f-26d5e16ee477 admin admin] PATH_INFO: `/v3/users/6a7eb6c39b754cd789c1e844422973af` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:58.273908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc84d891-a64f-4e73-940f-26d5e16ee477 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a7eb6c39b754cd789c1e844422973af)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:58.275916 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc84d891-a64f-4e73-940f-26d5e16ee477 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:58.364784 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-cc84d891-a64f-4e73-940f-26d5e16ee477 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a7eb6c39b754cd789c1e844422973af'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:58.373970 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-cc84d891-a64f-4e73-940f-26d5e16ee477 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a7eb6c39b754cd789c1e844422973af'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:58.401450 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-cc84d891-a64f-4e73-940f-26d5e16ee477 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a7eb6c39b754cd789c1e844422973af'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:58.402392 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-cc84d891-a64f-4e73-940f-26d5e16ee477 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a7eb6c39b754cd789c1e844422973af'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:58.418423 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-cc84d891-a64f-4e73-940f-26d5e16ee477 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a7eb6c39b754cd789c1e844422973af'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:58.432093 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1361/2708] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:20:58 2022] DELETE /identity/v3/users/6a7eb6c39b754cd789c1e844422973af => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:58.985375 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f5c3c-ce07-4fed-bf63-6dc23da1d42f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:58.987732 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-753f5c3c-ce07-4fed-bf63-6dc23da1d42f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:59.043721 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f5c3c-ce07-4fed-bf63-6dc23da1d42f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:59.158216 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1067f-b348-4e20-93db-ebd2b03c5d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:59.159276 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1067f-b348-4e20-93db-ebd2b03c5d26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:59.159663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1067f-b348-4e20-93db-ebd2b03c5d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:59.160048 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1067f-b348-4e20-93db-ebd2b03c5d26 admin admin] PATH_INFO: `/v3/projects/d7b7d463bbc34a06a48f40d4af548125` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:59.161417 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1067f-b348-4e20-93db-ebd2b03c5d26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7b7d463bbc34a06a48f40d4af548125)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:59.163534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1067f-b348-4e20-93db-ebd2b03c5d26 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:59.235300 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-22b1067f-b348-4e20-93db-ebd2b03c5d26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7b7d463bbc34a06a48f40d4af548125', 'request_id': 'req-22b1067f-b348-4e20-93db-ebd2b03c5d26'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:59.258018 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-22b1067f-b348-4e20-93db-ebd2b03c5d26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7b7d463bbc34a06a48f40d4af548125', 'request_id': 'req-22b1067f-b348-4e20-93db-ebd2b03c5d26'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:59.282568 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-22b1067f-b348-4e20-93db-ebd2b03c5d26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7b7d463bbc34a06a48f40d4af548125', 'request_id': 'req-22b1067f-b348-4e20-93db-ebd2b03c5d26'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:59.296794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-22b1067f-b348-4e20-93db-ebd2b03c5d26 admin admin] The token cache is being invalidate because project d7b7d463bbc34a06a48f40d4af548125 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:20:59.297162 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-22b1067f-b348-4e20-93db-ebd2b03c5d26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:59.297909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-22b1067f-b348-4e20-93db-ebd2b03c5d26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:20:59.304267 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1348/2709] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:20:58 2022] DELETE /identity/v3/projects/d7b7d463bbc34a06a48f40d4af548125 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:20:59.306349 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a81c22-dde9-4abe-ad34-4d883bacbd10 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:59.306852 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a81c22-dde9-4abe-ad34-4d883bacbd10 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:59.306968 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a81c22-dde9-4abe-ad34-4d883bacbd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:59.330932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-98a81c22-dde9-4abe-ad34-4d883bacbd10 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:59.332155 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-98a81c22-dde9-4abe-ad34-4d883bacbd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:59.410739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1362/2710] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:59.418835 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0df0e-c3c1-4f4c-a12c-1afd7d3563e9 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:59.419123 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0df0e-c3c1-4f4c-a12c-1afd7d3563e9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:59.419413 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0df0e-c3c1-4f4c-a12c-1afd7d3563e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:59.463682 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-eac0df0e-c3c1-4f4c-a12c-1afd7d3563e9 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:59.464952 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-eac0df0e-c3c1-4f4c-a12c-1afd7d3563e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:59.495971 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a81c22-dde9-4abe-ad34-4d883bacbd10 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:59.503998 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1349/2711] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:20:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:59.517882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a81c22-dde9-4abe-ad34-4d883bacbd10 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:59.541167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb0aee-2df9-4e47-9dd5-7f4b6a0607dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:59.542392 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb0aee-2df9-4e47-9dd5-7f4b6a0607dc admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:59.543497 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb0aee-2df9-4e47-9dd5-7f4b6a0607dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:59.544362 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb0aee-2df9-4e47-9dd5-7f4b6a0607dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:59.546383 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb0aee-2df9-4e47-9dd5-7f4b6a0607dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:20:59.546717 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb0aee-2df9-4e47-9dd5-7f4b6a0607dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:59.548454 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb0aee-2df9-4e47-9dd5-7f4b6a0607dc admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:59.584478 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1363/2712] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:20:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:20:59.589354 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0df0e-c3c1-4f4c-a12c-1afd7d3563e9 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:59.609082 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0df0e-c3c1-4f4c-a12c-1afd7d3563e9 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:59.622130 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c860dd-b72c-4785-8d78-b9f174ea7f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:59.622893 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c860dd-b72c-4785-8d78-b9f174ea7f56 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:59.623240 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c860dd-b72c-4785-8d78-b9f174ea7f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:59.623546 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c860dd-b72c-4785-8d78-b9f174ea7f56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:59.624309 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c860dd-b72c-4785-8d78-b9f174ea7f56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:59.628149 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c860dd-b72c-4785-8d78-b9f174ea7f56 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:59.707955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1350/2713] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:20:59 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:20:59.712796 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb0aee-2df9-4e47-9dd5-7f4b6a0607dc admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:59.736447 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb0aee-2df9-4e47-9dd5-7f4b6a0607dc admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:59.887448 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:59.902185 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f19d4-4216-499c-8029-b01a2bb3a54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:59.903055 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f19d4-4216-499c-8029-b01a2bb3a54c admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:59.903451 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f19d4-4216-499c-8029-b01a2bb3a54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:59.903832 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f19d4-4216-499c-8029-b01a2bb3a54c admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:59.904824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f19d4-4216-499c-8029-b01a2bb3a54c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:59.907826 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f19d4-4216-499c-8029-b01a2bb3a54c admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:59.957060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1364/2714] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:20:59 2022] POST /identity/v3/users => generated 290 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:20:59.961040 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c860dd-b72c-4785-8d78-b9f174ea7f56 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:59.998536 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c860dd-b72c-4785-8d78-b9f174ea7f56 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:00.014661 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e1139-88c0-49a3-b111-ceaf631ca399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:00.017754 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e1139-88c0-49a3-b111-ceaf631ca399 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:00.018151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e1139-88c0-49a3-b111-ceaf631ca399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:00.018511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e1139-88c0-49a3-b111-ceaf631ca399 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:00.019344 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e1139-88c0-49a3-b111-ceaf631ca399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:00.021349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e1139-88c0-49a3-b111-ceaf631ca399 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:00.055604 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1351/2715] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:21:00.059542 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f19d4-4216-499c-8029-b01a2bb3a54c admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:00.102169 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f19d4-4216-499c-8029-b01a2bb3a54c admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:00.113868 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956370e-b2b6-47b0-9af8-fb6b64425eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:00.116364 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956370e-b2b6-47b0-9af8-fb6b64425eed admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:00.116364 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956370e-b2b6-47b0-9af8-fb6b64425eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:00.116364 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956370e-b2b6-47b0-9af8-fb6b64425eed admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:00.120590 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0956370e-b2b6-47b0-9af8-fb6b64425eed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:00.130261 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0956370e-b2b6-47b0-9af8-fb6b64425eed admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:00.168431 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1365/2716] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:21:00.172223 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e1139-88c0-49a3-b111-ceaf631ca399 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:00.189170 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e1139-88c0-49a3-b111-ceaf631ca399 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:00.211462 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e31724-fcb7-4c69-b590-eed2ec913934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:00.212495 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e31724-fcb7-4c69-b590-eed2ec913934 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:00.213049 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e31724-fcb7-4c69-b590-eed2ec913934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:00.213546 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e31724-fcb7-4c69-b590-eed2ec913934 admin admin] PATH_INFO: `/v3/projects/b77126371c5d44db8ab75fbfca0737af/users/373d58e9f0e6472dad412ac02bff2b94/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:00.229648 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:21:00.272101 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e31724-fcb7-4c69-b590-eed2ec913934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b77126371c5d44db8ab75fbfca0737af, user_id=373d58e9f0e6472dad412ac02bff2b94, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:00.275040 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e31724-fcb7-4c69-b590-eed2ec913934 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:00.308406 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1352/2717] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:21:00 2022] PUT /identity/v3/projects/b77126371c5d44db8ab75fbfca0737af/users/373d58e9f0e6472dad412ac02bff2b94/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:21:00.316031 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5a86b-4780-4c09-8044-a135fb30a72c None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:00.316031 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5a86b-4780-4c09-8044-a135fb30a72c None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:00.316031 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5a86b-4780-4c09-8044-a135fb30a72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:00.349058 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-afe5a86b-4780-4c09-8044-a135fb30a72c None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:21:00.350421 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-afe5a86b-4780-4c09-8044-a135fb30a72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:00.461623 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1366/2718] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:21:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:21:00.473466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e31724-fcb7-4c69-b590-eed2ec913934 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:00.478965 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d9e31724-fcb7-4c69-b590-eed2ec913934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:00.604117 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e31724-fcb7-4c69-b590-eed2ec913934 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:00.608659 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94cf68d-7aa0-4dc8-89c3-746f31b88784 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:00.610292 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cf68d-7aa0-4dc8-89c3-746f31b88784 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:00.610600 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cf68d-7aa0-4dc8-89c3-746f31b88784 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:00.610936 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cf68d-7aa0-4dc8-89c3-746f31b88784 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:00.626007 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94cf68d-7aa0-4dc8-89c3-746f31b88784 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:00.627382 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94cf68d-7aa0-4dc8-89c3-746f31b88784 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:00.648394 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1353/2719] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:21:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:21:04.387708 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61817d3a-d55b-43c0-aeaf-84233273b97c None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:04.388819 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61817d3a-d55b-43c0-aeaf-84233273b97c None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:04.388819 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61817d3a-d55b-43c0-aeaf-84233273b97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:04.452333 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-61817d3a-d55b-43c0-aeaf-84233273b97c None None] MFA Rules not processed for user `2eeb3db00b9d4c3ba667bd25abe72b27`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:21:04.458832 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-61817d3a-d55b-43c0-aeaf-84233273b97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:04.640253 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1367/2720] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:21:04 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:21:04.648141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94cf68d-7aa0-4dc8-89c3-746f31b88784 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:04.650034 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e94cf68d-7aa0-4dc8-89c3-746f31b88784 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:04.858155 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94cf68d-7aa0-4dc8-89c3-746f31b88784 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:04.879299 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0eeb7f-1e64-4498-aff3-7e5c93d23f25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:04.880336 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0eeb7f-1e64-4498-aff3-7e5c93d23f25 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:04.880787 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0eeb7f-1e64-4498-aff3-7e5c93d23f25 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:04.881229 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0eeb7f-1e64-4498-aff3-7e5c93d23f25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:04.883407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3b0eeb7f-1e64-4498-aff3-7e5c93d23f25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:04.994219 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0eeb7f-1e64-4498-aff3-7e5c93d23f25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:04.996317 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0eeb7f-1e64-4498-aff3-7e5c93d23f25 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:05.011965 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1354/2721] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:21:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:21:06.944640 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e327a-cf13-4fdd-892d-9639d026eab8 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:06.945237 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e327a-cf13-4fdd-892d-9639d026eab8 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:06.945237 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e327a-cf13-4fdd-892d-9639d026eab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:06.981490 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-0f5e327a-cf13-4fdd-892d-9639d026eab8 None None] MFA Rules not processed for user `080c7129a52e49c698cb2dc4afe14de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:21:06.983637 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-0f5e327a-cf13-4fdd-892d-9639d026eab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:06.999046 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1368/2722] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:21:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:21:09.620622 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ed530-4943-4843-ad46-580980035c87 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:09.622009 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ed530-4943-4843-ad46-580980035c87 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:09.622009 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ed530-4943-4843-ad46-580980035c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:09.656463 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b11ed530-4943-4843-ad46-580980035c87 None None] MFA Rules not processed for user `373d58e9f0e6472dad412ac02bff2b94`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:21:09.658046 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b11ed530-4943-4843-ad46-580980035c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:09.678830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:21:09.767348 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1355/2723] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:21:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:21:09.772239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e327a-cf13-4fdd-892d-9639d026eab8 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:09.812097 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e327a-cf13-4fdd-892d-9639d026eab8 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:09.852277 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2bb3a1-037b-4f7b-8dda-db851773cf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:09.853392 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bb3a1-037b-4f7b-8dda-db851773cf39 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:09.853845 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bb3a1-037b-4f7b-8dda-db851773cf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:09.854342 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bb3a1-037b-4f7b-8dda-db851773cf39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:09.855242 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2bb3a1-037b-4f7b-8dda-db851773cf39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:09.858015 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2bb3a1-037b-4f7b-8dda-db851773cf39 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:09.922281 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1369/2724] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:21:09 2022] POST /identity/v3/projects => generated 379 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:21:09.927159 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ed530-4943-4843-ad46-580980035c87 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:09.941906 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ed530-4943-4843-ad46-580980035c87 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:10.096839 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e313a-a73f-4955-a6dc-cf0e21b44e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:10.097540 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e313a-a73f-4955-a6dc-cf0e21b44e1f admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:10.097889 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e313a-a73f-4955-a6dc-cf0e21b44e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:10.098214 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e313a-a73f-4955-a6dc-cf0e21b44e1f admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:10.098926 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1e313a-a73f-4955-a6dc-cf0e21b44e1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:10.100879 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1e313a-a73f-4955-a6dc-cf0e21b44e1f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:10.132109 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1356/2725] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:21:09 2022] POST /identity/v3/users => generated 296 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:21:10.137246 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2bb3a1-037b-4f7b-8dda-db851773cf39 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:10.149740 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2bb3a1-037b-4f7b-8dda-db851773cf39 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:10.165343 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad1055-4087-4ee8-897f-1137c6ee4d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:10.166512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad1055-4087-4ee8-897f-1137c6ee4d56 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:10.167199 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad1055-4087-4ee8-897f-1137c6ee4d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:10.168285 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad1055-4087-4ee8-897f-1137c6ee4d56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:10.170324 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad1055-4087-4ee8-897f-1137c6ee4d56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:10.173963 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad1055-4087-4ee8-897f-1137c6ee4d56 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:10.195794 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1370/2726] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:21:10 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:21:10.202282 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e313a-a73f-4955-a6dc-cf0e21b44e1f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:10.241262 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e313a-a73f-4955-a6dc-cf0e21b44e1f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:10.258877 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b1623-31fc-4078-bdb6-d7424ffa2887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:10.260317 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b1623-31fc-4078-bdb6-d7424ffa2887 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:10.260711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b1623-31fc-4078-bdb6-d7424ffa2887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:10.261374 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b1623-31fc-4078-bdb6-d7424ffa2887 admin admin] PATH_INFO: `/v3/projects/4be9ac41669a4f3194ef165f3fc5cb31/users/f8a60b97a4bb4749a145304edb0b07cf/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:10.316565 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b1623-31fc-4078-bdb6-d7424ffa2887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4be9ac41669a4f3194ef165f3fc5cb31, user_id=f8a60b97a4bb4749a145304edb0b07cf, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:10.320528 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b1623-31fc-4078-bdb6-d7424ffa2887 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:10.357937 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1357/2727] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:21:10 2022] PUT /identity/v3/projects/4be9ac41669a4f3194ef165f3fc5cb31/users/f8a60b97a4bb4749a145304edb0b07cf/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:21:10.359241 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad1055-4087-4ee8-897f-1137c6ee4d56 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:10.373544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad1055-4087-4ee8-897f-1137c6ee4d56 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:10.395476 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:21:10.515114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54c62b-d9cc-42ba-a60b-9781842476af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:10.515931 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54c62b-d9cc-42ba-a60b-9781842476af admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:10.516289 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54c62b-d9cc-42ba-a60b-9781842476af admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:10.516690 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54c62b-d9cc-42ba-a60b-9781842476af admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:10.517499 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54c62b-d9cc-42ba-a60b-9781842476af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:10.519246 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54c62b-d9cc-42ba-a60b-9781842476af admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:10.534988 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1371/2728] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:21:10 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:21:10.539414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b1623-31fc-4078-bdb6-d7424ffa2887 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:10.564236 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b1623-31fc-4078-bdb6-d7424ffa2887 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:10.576278 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eede200-31ec-4e15-a022-448fdd8bccda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:10.578117 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eede200-31ec-4e15-a022-448fdd8bccda admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:10.578411 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eede200-31ec-4e15-a022-448fdd8bccda admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:10.578691 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eede200-31ec-4e15-a022-448fdd8bccda admin admin] PATH_INFO: `/v3/projects/4be9ac41669a4f3194ef165f3fc5cb31/users/f8a60b97a4bb4749a145304edb0b07cf/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:10.580857 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eede200-31ec-4e15-a022-448fdd8bccda admin admin] RBAC: Authorizing `identity:create_grant(project_id=4be9ac41669a4f3194ef165f3fc5cb31, user_id=f8a60b97a4bb4749a145304edb0b07cf, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:10.590786 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eede200-31ec-4e15-a022-448fdd8bccda admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:10.603924 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1358/2729] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:21:10 2022] PUT /identity/v3/projects/4be9ac41669a4f3194ef165f3fc5cb31/users/f8a60b97a4bb4749a145304edb0b07cf/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:21:18.002673 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62ad51-f391-4c33-a607-9da2b0765639 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:18.006358 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62ad51-f391-4c33-a607-9da2b0765639 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:18.007153 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62ad51-f391-4c33-a607-9da2b0765639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:18.066970 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-7c62ad51-f391-4c33-a607-9da2b0765639 None None] MFA Rules not processed for user `f8a60b97a4bb4749a145304edb0b07cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:21:18.068688 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-7c62ad51-f391-4c33-a607-9da2b0765639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:18.223763 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1372/2730] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:21:17 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:21:18.238339 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eede200-31ec-4e15-a022-448fdd8bccda admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:18.392125 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eede200-31ec-4e15-a022-448fdd8bccda admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:18.400134 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1198ce3-b4c0-4a47-b443-e0dc35887ab2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:18.400839 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1198ce3-b4c0-4a47-b443-e0dc35887ab2 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:18.401132 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1198ce3-b4c0-4a47-b443-e0dc35887ab2 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:18.401418 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1198ce3-b4c0-4a47-b443-e0dc35887ab2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:18.407962 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:21:18.535757 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1198ce3-b4c0-4a47-b443-e0dc35887ab2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:18.538528 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1198ce3-b4c0-4a47-b443-e0dc35887ab2 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:18.551689 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1359/2731] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:21:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:21:18.632345 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5423dd1-5390-47aa-9d27-1383e9468a5e None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:18.633618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5423dd1-5390-47aa-9d27-1383e9468a5e None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:18.633618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5423dd1-5390-47aa-9d27-1383e9468a5e None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:18.639526 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1373/2732] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jun 16 16:21:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:21:18.645620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d81137-1527-46d9-8def-c3cf5defac9b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:18.645620 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d81137-1527-46d9-8def-c3cf5defac9b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:18.646156 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d81137-1527-46d9-8def-c3cf5defac9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:18.697736 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-71d81137-1527-46d9-8def-c3cf5defac9b None None] MFA Rules not processed for user `c3645b1a0be34c01ac87c734fcd85ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:21:18.698896 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-71d81137-1527-46d9-8def-c3cf5defac9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:18.719699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1360/2733] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jun 16 16:21:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:21:18.734500 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5423dd1-5390-47aa-9d27-1383e9468a5e None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:18.736536 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-a5423dd1-5390-47aa-9d27-1383e9468a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:18.885868 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5423dd1-5390-47aa-9d27-1383e9468a5e None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:18.907043 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1544b7-7a51-4398-9aad-0ea7fc2fabe5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f11beca9c5148eea4cc45f8dc89dbda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:18.908445 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1544b7-7a51-4398-9aad-0ea7fc2fabe5 service glance] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:18.908997 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1544b7-7a51-4398-9aad-0ea7fc2fabe5 service glance] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:18.909288 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1544b7-7a51-4398-9aad-0ea7fc2fabe5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:18.926238 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1544b7-7a51-4398-9aad-0ea7fc2fabe5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:18.927741 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1544b7-7a51-4398-9aad-0ea7fc2fabe5 service glance] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:18.957092 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1374/2734] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:21:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:21:19.090503 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e474988-0cd9-4070-8922-84184c9c9462 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:19.090803 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e474988-0cd9-4070-8922-84184c9c9462 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:19.091225 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e474988-0cd9-4070-8922-84184c9c9462 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:19.093148 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1361/2735] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jun 16 16:21:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:21:19.100939 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2e038-7a03-4c17-954b-c33b979e20a5 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:19.101819 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2e038-7a03-4c17-954b-c33b979e20a5 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:19.102245 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2e038-7a03-4c17-954b-c33b979e20a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:19.160505 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-3de2e038-7a03-4c17-954b-c33b979e20a5 None None] MFA Rules not processed for user `5a84f8aed50c4b3e8e5b33c5b587b26e`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:21:19.161964 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-3de2e038-7a03-4c17-954b-c33b979e20a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:19.221788 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=100 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:21:19.243026 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1375/2736] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jun 16 16:21:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:21:19.264469 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e474988-0cd9-4070-8922-84184c9c9462 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:19.266951 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7e474988-0cd9-4070-8922-84184c9c9462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:19.285634 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e474988-0cd9-4070-8922-84184c9c9462 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:19.290405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c847dd-f974-45ed-9e38-cd8df6fd3d82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:19.291073 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c847dd-f974-45ed-9e38-cd8df6fd3d82 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:19.291357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c847dd-f974-45ed-9e38-cd8df6fd3d82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:19.291764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c847dd-f974-45ed-9e38-cd8df6fd3d82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:19.300684 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c847dd-f974-45ed-9e38-cd8df6fd3d82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:19.302733 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c847dd-f974-45ed-9e38-cd8df6fd3d82 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:19.311684 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1362/2737] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:21:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:21:19.420471 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74664e-2dff-47b3-9279-f6c106afbab5 None None] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:19.420764 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74664e-2dff-47b3-9279-f6c106afbab5 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:19.421052 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74664e-2dff-47b3-9279-f6c106afbab5 None None] PATH_INFO: `/` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:19.422911 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1376/2738] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jun 16 16:21:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:21:19.429361 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418af267-d3fb-4368-954d-31959bb0d1fa None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:19.430103 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418af267-d3fb-4368-954d-31959bb0d1fa None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:19.430495 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418af267-d3fb-4368-954d-31959bb0d1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:19.450131 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-418af267-d3fb-4368-954d-31959bb0d1fa None None] MFA Rules not processed for user `c3645b1a0be34c01ac87c734fcd85ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:21:19.452067 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-418af267-d3fb-4368-954d-31959bb0d1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:19.464686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1363/2739] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jun 16 16:21:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:21:19.479175 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d74664e-2dff-47b3-9279-f6c106afbab5 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:19.501183 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d74664e-2dff-47b3-9279-f6c106afbab5 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:19.515217 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69491230-554e-41ea-b34d-8c3938c8f95c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:19.515896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69491230-554e-41ea-b34d-8c3938c8f95c service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:19.516186 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69491230-554e-41ea-b34d-8c3938c8f95c service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:19.516498 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69491230-554e-41ea-b34d-8c3938c8f95c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:19.528247 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69491230-554e-41ea-b34d-8c3938c8f95c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:19.530856 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69491230-554e-41ea-b34d-8c3938c8f95c service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:19.545362 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1377/2740] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:21:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:21:22.327337 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec1854-1944-4021-960d-0b257ad1d429 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:22.327822 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec1854-1944-4021-960d-0b257ad1d429 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:22.328078 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec1854-1944-4021-960d-0b257ad1d429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:22.351114 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-10ec1854-1944-4021-960d-0b257ad1d429 None None] MFA Rules not processed for user `2c2e0e19880f4b048faf034dd38c7365`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:21:22.352398 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-10ec1854-1944-4021-960d-0b257ad1d429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:22.434723 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1364/2741] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 16:21:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:21:22.448961 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69491230-554e-41ea-b34d-8c3938c8f95c service neutron] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:22.450829 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-69491230-554e-41ea-b34d-8c3938c8f95c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:22.559318 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69491230-554e-41ea-b34d-8c3938c8f95c service neutron] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:22.574765 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601cde7-df93-4619-bb9f-04f45335d5d0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0082e37ce54f49f7837601349508c89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:22.576389 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601cde7-df93-4619-bb9f-04f45335d5d0 service swift] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:22.576763 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601cde7-df93-4619-bb9f-04f45335d5d0 service swift] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:22.577044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601cde7-df93-4619-bb9f-04f45335d5d0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:22.588060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e601cde7-df93-4619-bb9f-04f45335d5d0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:22.590130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e601cde7-df93-4619-bb9f-04f45335d5d0 service swift] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:22.605996 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1378/2742] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 16:21:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:21:28.335333 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec1854-1944-4021-960d-0b257ad1d429 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:28.349842 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec1854-1944-4021-960d-0b257ad1d429 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:28.354164 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f80decc-695d-4796-b6e0-2282fb1f4081 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:28.354842 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80decc-695d-4796-b6e0-2282fb1f4081 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:28.355132 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80decc-695d-4796-b6e0-2282fb1f4081 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:28.355414 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80decc-695d-4796-b6e0-2282fb1f4081 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:28.358140 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8f80decc-695d-4796-b6e0-2282fb1f4081 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:28.395363 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f80decc-695d-4796-b6e0-2282fb1f4081 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:28.397894 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f80decc-695d-4796-b6e0-2282fb1f4081 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:28.413277 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1365/2743] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:21:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:21:32.600911 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=43 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:21:38.408684 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=74 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:21:39.317278 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd72f2-4772-4ab6-9bcc-5b07de2bf35b None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:39.317843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd72f2-4772-4ab6-9bcc-5b07de2bf35b None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:39.317843 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd72f2-4772-4ab6-9bcc-5b07de2bf35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:39.336128 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-a9dd72f2-4772-4ab6-9bcc-5b07de2bf35b None None] MFA Rules not processed for user `373d58e9f0e6472dad412ac02bff2b94`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:21:39.337338 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-a9dd72f2-4772-4ab6-9bcc-5b07de2bf35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:39.345166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1379/2744] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:21:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:21:40.898550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f80decc-695d-4796-b6e0-2282fb1f4081 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:40.908059 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f80decc-695d-4796-b6e0-2282fb1f4081 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:40.912442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ccb850-e332-4ee3-a9cb-3700c7757b94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:40.913141 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ccb850-e332-4ee3-a9cb-3700c7757b94 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:40.913536 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ccb850-e332-4ee3-a9cb-3700c7757b94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:40.913868 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ccb850-e332-4ee3-a9cb-3700c7757b94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:40.915872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d6ccb850-e332-4ee3-a9cb-3700c7757b94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:40.930287 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ccb850-e332-4ee3-a9cb-3700c7757b94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:40.931736 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ccb850-e332-4ee3-a9cb-3700c7757b94 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:40.944566 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1366/2745] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:21:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:21:45.323142 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4693f8-9d88-48ec-b156-6a4e57c332a5 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:45.324584 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4693f8-9d88-48ec-b156-6a4e57c332a5 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:45.324584 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4693f8-9d88-48ec-b156-6a4e57c332a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:45.348986 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-bc4693f8-9d88-48ec-b156-6a4e57c332a5 None None] MFA Rules not processed for user `2c2e0e19880f4b048faf034dd38c7365`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:21:45.350402 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-bc4693f8-9d88-48ec-b156-6a4e57c332a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:45.358586 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1380/2746] 10.222.0.17 () {58 vars in 967 bytes} [Thu Jun 16 16:21:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:21:45.382621 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ccb850-e332-4ee3-a9cb-3700c7757b94 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:45.396670 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ccb850-e332-4ee3-a9cb-3700c7757b94 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:45.405355 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b726b-0c8f-468f-afef-ad2ec060897f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0082e37ce54f49f7837601349508c89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:45.406082 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b726b-0c8f-468f-afef-ad2ec060897f service swift] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:45.406374 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b726b-0c8f-468f-afef-ad2ec060897f service swift] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:45.406654 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b726b-0c8f-468f-afef-ad2ec060897f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:45.416333 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304b726b-0c8f-468f-afef-ad2ec060897f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:45.417912 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304b726b-0c8f-468f-afef-ad2ec060897f service swift] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:45.433304 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1367/2747] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 16:21:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 16:21:45.874882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4693f8-9d88-48ec-b156-6a4e57c332a5 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:45.887257 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4693f8-9d88-48ec-b156-6a4e57c332a5 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:45.891983 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de815f-90ca-4548-810a-d09a5bb9d916 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:45.892738 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de815f-90ca-4548-810a-d09a5bb9d916 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:45.893072 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de815f-90ca-4548-810a-d09a5bb9d916 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:45.893527 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de815f-90ca-4548-810a-d09a5bb9d916 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:45.895860 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-24de815f-90ca-4548-810a-d09a5bb9d916 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:45.923245 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de815f-90ca-4548-810a-d09a5bb9d916 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:45.925004 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de815f-90ca-4548-810a-d09a5bb9d916 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:45.944123 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1381/2748] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:21:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:21:48.682219 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b726b-0c8f-468f-afef-ad2ec060897f service swift] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:48.698504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b726b-0c8f-468f-afef-ad2ec060897f service swift] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:48.704446 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb65db4-db96-4c6f-b9f0-91eae0ff7a89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:48.705511 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb65db4-db96-4c6f-b9f0-91eae0ff7a89 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:48.706032 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb65db4-db96-4c6f-b9f0-91eae0ff7a89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:48.706515 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb65db4-db96-4c6f-b9f0-91eae0ff7a89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:48.809774 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb65db4-db96-4c6f-b9f0-91eae0ff7a89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:48.812588 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb65db4-db96-4c6f-b9f0-91eae0ff7a89 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:48.829677 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1368/2749] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:21:48 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:21:55.932888 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=12 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:21:58.727369 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de815f-90ca-4548-810a-d09a5bb9d916 service neutron] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:58.737584 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de815f-90ca-4548-810a-d09a5bb9d916 service neutron] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:58.741517 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09718f5-60d2-49c9-9736-09ea463ef915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:58.742210 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09718f5-60d2-49c9-9736-09ea463ef915 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:58.742496 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09718f5-60d2-49c9-9736-09ea463ef915 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:58.742805 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09718f5-60d2-49c9-9736-09ea463ef915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:58.753089 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09718f5-60d2-49c9-9736-09ea463ef915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:58.754582 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09718f5-60d2-49c9-9736-09ea463ef915 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:58.765337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1382/2750] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:21:58 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:21:58.821698 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=42 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:22:08.760633 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=6 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:22:36.436127 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8552cb6-5d6c-454f-9131-15cf25460d07 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:22:36.436763 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8552cb6-5d6c-454f-9131-15cf25460d07 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:22:36.436763 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8552cb6-5d6c-454f-9131-15cf25460d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:22:36.455941 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f8552cb6-5d6c-454f-9131-15cf25460d07 None None] MFA Rules not processed for user `2b32b74cc1e44ce5a7e687e3c04e8099`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:22:36.470911 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f8552cb6-5d6c-454f-9131-15cf25460d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:22:36.530493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1369/2751] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:22:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:22:39.051866 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09718f5-60d2-49c9-9736-09ea463ef915 service neutron] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:22:39.061769 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09718f5-60d2-49c9-9736-09ea463ef915 service neutron] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:22:39.066129 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2079ad-7de3-455e-ac97-4ed12d067760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:22:39.066788 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2079ad-7de3-455e-ac97-4ed12d067760 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:22:39.067447 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2079ad-7de3-455e-ac97-4ed12d067760 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:22:39.067447 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2079ad-7de3-455e-ac97-4ed12d067760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:22:39.069355 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-eb2079ad-7de3-455e-ac97-4ed12d067760 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:22:39.081642 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2079ad-7de3-455e-ac97-4ed12d067760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:22:39.083170 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2079ad-7de3-455e-ac97-4ed12d067760 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:22:39.093812 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1383/2752] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:22:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:22:46.522430 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=25 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:22:49.088178 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=6 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:23:45.059357 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2ed63-f88d-422b-ae9a-595a5355a564 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:23:45.059895 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2ed63-f88d-422b-ae9a-595a5355a564 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:23:45.059895 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2ed63-f88d-422b-ae9a-595a5355a564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:23:45.078692 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-6ec2ed63-f88d-422b-ae9a-595a5355a564 None None] MFA Rules not processed for user `2c2e0e19880f4b048faf034dd38c7365`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:23:45.080508 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6ec2ed63-f88d-422b-ae9a-595a5355a564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:23:45.088424 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1370/2753] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 16:23:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:23:45.138216 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2079ad-7de3-455e-ac97-4ed12d067760 service neutron] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:23:45.151921 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2079ad-7de3-455e-ac97-4ed12d067760 service neutron] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:23:45.161912 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca5351-4592-421d-adbe-e7c6c7ae0bc8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0082e37ce54f49f7837601349508c89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:23:45.162664 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca5351-4592-421d-adbe-e7c6c7ae0bc8 service swift] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:23:45.162909 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca5351-4592-421d-adbe-e7c6c7ae0bc8 service swift] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:23:45.163190 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca5351-4592-421d-adbe-e7c6c7ae0bc8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:23:45.173006 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca5351-4592-421d-adbe-e7c6c7ae0bc8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:23:45.174696 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca5351-4592-421d-adbe-e7c6c7ae0bc8 service swift] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:23:45.184119 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1384/2754] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 16:23:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:23:45.778132 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec2ed63-f88d-422b-ae9a-595a5355a564 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:23:45.796884 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec2ed63-f88d-422b-ae9a-595a5355a564 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:23:45.807161 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c728c8-ecb5-414d-a9fe-d9a3e4a73583 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:23:45.807851 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c728c8-ecb5-414d-a9fe-d9a3e4a73583 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:23:45.808145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c728c8-ecb5-414d-a9fe-d9a3e4a73583 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:23:45.808500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c728c8-ecb5-414d-a9fe-d9a3e4a73583 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:23:45.810520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a6c728c8-ecb5-414d-a9fe-d9a3e4a73583 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:23:45.826534 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c728c8-ecb5-414d-a9fe-d9a3e4a73583 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:23:45.827676 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c728c8-ecb5-414d-a9fe-d9a3e4a73583 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:23:45.844764 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1371/2755] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:23:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:23:55.181227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=6 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:23:55.836703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=9 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:23:59.984756 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca5351-4592-421d-adbe-e7c6c7ae0bc8 service swift] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:23:59.986334 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-c0ca5351-4592-421d-adbe-e7c6c7ae0bc8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:00.068797 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca5351-4592-421d-adbe-e7c6c7ae0bc8 service swift] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:00.073872 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70514a86-7e0a-44ad-92f9-0f277a247470 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36b63614297346e29ff276eab444fe5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:00.074545 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70514a86-7e0a-44ad-92f9-0f277a247470 service placement] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:00.074826 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70514a86-7e0a-44ad-92f9-0f277a247470 service placement] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:00.075097 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70514a86-7e0a-44ad-92f9-0f277a247470 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:00.076883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-70514a86-7e0a-44ad-92f9-0f277a247470 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:00.090254 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70514a86-7e0a-44ad-92f9-0f277a247470 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:00.091878 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70514a86-7e0a-44ad-92f9-0f277a247470 service placement] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:00.104055 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1385/2756] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:23:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:24:01.916345 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb58c680-50d0-4416-a3e7-26420ab2a796 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:01.916810 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb58c680-50d0-4416-a3e7-26420ab2a796 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:01.916905 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb58c680-50d0-4416-a3e7-26420ab2a796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:01.935957 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-eb58c680-50d0-4416-a3e7-26420ab2a796 None None] MFA Rules not processed for user `2c2e0e19880f4b048faf034dd38c7365`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:24:01.937163 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-eb58c680-50d0-4416-a3e7-26420ab2a796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:01.945351 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1372/2757] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 16:24:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:24:01.958633 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70514a86-7e0a-44ad-92f9-0f277a247470 service placement] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:01.970524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70514a86-7e0a-44ad-92f9-0f277a247470 service placement] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:01.975620 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0ca76-c36a-435b-afb7-2d584b79d9fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0082e37ce54f49f7837601349508c89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:01.976269 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0ca76-c36a-435b-afb7-2d584b79d9fe service swift] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:01.976551 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0ca76-c36a-435b-afb7-2d584b79d9fe service swift] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:01.976831 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0ca76-c36a-435b-afb7-2d584b79d9fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:01.986057 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0ca76-c36a-435b-afb7-2d584b79d9fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:01.987766 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0ca76-c36a-435b-afb7-2d584b79d9fe service swift] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:01.996428 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1386/2758] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 16:24:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:24:11.931140 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:11.993185 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=38 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:14.396561 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb58c680-50d0-4416-a3e7-26420ab2a796 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:14.411741 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb58c680-50d0-4416-a3e7-26420ab2a796 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:14.423028 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeafcd5f-51c1-49d4-b1ae-dd8733f2dca2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:14.424317 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeafcd5f-51c1-49d4-b1ae-dd8733f2dca2 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:14.424462 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeafcd5f-51c1-49d4-b1ae-dd8733f2dca2 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:14.425421 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeafcd5f-51c1-49d4-b1ae-dd8733f2dca2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:14.426926 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-aeafcd5f-51c1-49d4-b1ae-dd8733f2dca2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:14.441591 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeafcd5f-51c1-49d4-b1ae-dd8733f2dca2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:14.443630 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeafcd5f-51c1-49d4-b1ae-dd8733f2dca2 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:14.456726 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1373/2759] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:24:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:24:15.824493 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4c60c-b09d-434a-b3cc-c3882314747a None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:15.825315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4c60c-b09d-434a-b3cc-c3882314747a None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:15.825315 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4c60c-b09d-434a-b3cc-c3882314747a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:15.857835 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-d2a4c60c-b09d-434a-b3cc-c3882314747a None None] MFA Rules not processed for user `373d58e9f0e6472dad412ac02bff2b94`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:24:15.861797 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-d2a4c60c-b09d-434a-b3cc-c3882314747a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:15.874631 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1387/2760] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:24:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:24:16.260535 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeafcd5f-51c1-49d4-b1ae-dd8733f2dca2 service nova] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:16.271400 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeafcd5f-51c1-49d4-b1ae-dd8733f2dca2 service nova] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:16.275341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990ba3a-4e08-4671-955b-487bd67c6030 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:16.276001 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990ba3a-4e08-4671-955b-487bd67c6030 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:16.276291 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990ba3a-4e08-4671-955b-487bd67c6030 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:16.276570 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990ba3a-4e08-4671-955b-487bd67c6030 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:16.278489 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3990ba3a-4e08-4671-955b-487bd67c6030 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:16.291326 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990ba3a-4e08-4671-955b-487bd67c6030 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:16.294223 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990ba3a-4e08-4671-955b-487bd67c6030 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:16.310042 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1374/2761] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:24:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:24:18.031894 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51053037-09e8-4072-b9cd-85c5a22d3b8f None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:18.033349 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51053037-09e8-4072-b9cd-85c5a22d3b8f None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:18.033957 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51053037-09e8-4072-b9cd-85c5a22d3b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:18.058163 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-51053037-09e8-4072-b9cd-85c5a22d3b8f None None] MFA Rules not processed for user `2c2e0e19880f4b048faf034dd38c7365`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:24:18.059396 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-51053037-09e8-4072-b9cd-85c5a22d3b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:18.068850 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1388/2762] 10.222.0.17 () {58 vars in 967 bytes} [Thu Jun 16 16:24:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:24:18.082218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990ba3a-4e08-4671-955b-487bd67c6030 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:18.097507 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990ba3a-4e08-4671-955b-487bd67c6030 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:18.106325 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f5ff7-f2ce-43e7-abcc-922f65b77e96 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0082e37ce54f49f7837601349508c89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:18.107464 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f5ff7-f2ce-43e7-abcc-922f65b77e96 service swift] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:18.108153 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f5ff7-f2ce-43e7-abcc-922f65b77e96 service swift] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:18.108470 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f5ff7-f2ce-43e7-abcc-922f65b77e96 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:18.124053 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2f5ff7-f2ce-43e7-abcc-922f65b77e96 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:18.125866 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2f5ff7-f2ce-43e7-abcc-922f65b77e96 service swift] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:18.137168 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1375/2763] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 16:24:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:24:28.052498 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=6 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:28.131721 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=18 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:34.796201 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51053037-09e8-4072-b9cd-85c5a22d3b8f None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:34.856737 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51053037-09e8-4072-b9cd-85c5a22d3b8f None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:34.865521 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52351744-d249-4f68-9059-5ef837596640 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:34.866615 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52351744-d249-4f68-9059-5ef837596640 service nova] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:34.867093 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52351744-d249-4f68-9059-5ef837596640 service nova] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:34.867561 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52351744-d249-4f68-9059-5ef837596640 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:34.871054 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-52351744-d249-4f68-9059-5ef837596640 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:34.917246 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52351744-d249-4f68-9059-5ef837596640 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:34.918986 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52351744-d249-4f68-9059-5ef837596640 service nova] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:34.934164 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1389/2764] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:24:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:24:44.927368 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=28 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:59.022184 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f5ff7-f2ce-43e7-abcc-922f65b77e96 service swift] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:59.024388 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-df2f5ff7-f2ce-43e7-abcc-922f65b77e96 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:59.171569 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f5ff7-f2ce-43e7-abcc-922f65b77e96 service swift] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:59.175841 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8af458-53e1-4528-ad71-191a8291ca70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:59.177140 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8af458-53e1-4528-ad71-191a8291ca70 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:59.177140 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8af458-53e1-4528-ad71-191a8291ca70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:59.177140 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8af458-53e1-4528-ad71-191a8291ca70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:59.179045 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6e8af458-53e1-4528-ad71-191a8291ca70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:59.191874 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8af458-53e1-4528-ad71-191a8291ca70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:59.193341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8af458-53e1-4528-ad71-191a8291ca70 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:59.204834 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1376/2765] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:24:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:02.792579 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52351744-d249-4f68-9059-5ef837596640 service nova] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:02.794710 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-52351744-d249-4f68-9059-5ef837596640 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:02.845027 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52351744-d249-4f68-9059-5ef837596640 service nova] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:02.852037 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb324c29-6525-403e-92ec-30271879fcc3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36b63614297346e29ff276eab444fe5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:02.852849 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb324c29-6525-403e-92ec-30271879fcc3 service placement] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:02.853242 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb324c29-6525-403e-92ec-30271879fcc3 service placement] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:02.853580 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb324c29-6525-403e-92ec-30271879fcc3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:02.856184 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-eb324c29-6525-403e-92ec-30271879fcc3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:02.870999 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb324c29-6525-403e-92ec-30271879fcc3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:02.872760 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb324c29-6525-403e-92ec-30271879fcc3 service placement] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:02.888436 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1390/2766] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:06.740771 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8af458-53e1-4528-ad71-191a8291ca70 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:06.761515 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8af458-53e1-4528-ad71-191a8291ca70 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:06.765431 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aaf2f3-7655-463a-aeee-4e2b6081c956 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:06.766114 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aaf2f3-7655-463a-aeee-4e2b6081c956 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:06.766405 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aaf2f3-7655-463a-aeee-4e2b6081c956 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:06.766686 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aaf2f3-7655-463a-aeee-4e2b6081c956 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:06.768537 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d6aaf2f3-7655-463a-aeee-4e2b6081c956 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:06.862960 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aaf2f3-7655-463a-aeee-4e2b6081c956 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:06.864393 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aaf2f3-7655-463a-aeee-4e2b6081c956 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:06.883727 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1377/2767] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:10.138351 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb324c29-6525-403e-92ec-30271879fcc3 service placement] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:10.153722 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb324c29-6525-403e-92ec-30271879fcc3 service placement] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:10.164483 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe686473-ce2f-4da5-97d7-baad0de1c1db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36b63614297346e29ff276eab444fe5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:10.168697 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe686473-ce2f-4da5-97d7-baad0de1c1db service placement] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:10.170565 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe686473-ce2f-4da5-97d7-baad0de1c1db service placement] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:10.171452 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe686473-ce2f-4da5-97d7-baad0de1c1db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:10.176408 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-fe686473-ce2f-4da5-97d7-baad0de1c1db service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:10.194679 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe686473-ce2f-4da5-97d7-baad0de1c1db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:10.196265 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe686473-ce2f-4da5-97d7-baad0de1c1db service placement] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:10.208026 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1391/2768] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:10.217359 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aaf2f3-7655-463a-aeee-4e2b6081c956 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:10.219673 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d6aaf2f3-7655-463a-aeee-4e2b6081c956 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:10.234669 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aaf2f3-7655-463a-aeee-4e2b6081c956 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:10.239579 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e041288-d70a-44ef-856c-c1ad71e04d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:10.240761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e041288-d70a-44ef-856c-c1ad71e04d2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:10.241240 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e041288-d70a-44ef-856c-c1ad71e04d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:10.241769 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e041288-d70a-44ef-856c-c1ad71e04d2a admin admin] PATH_INFO: `/v3/users/2b32b74cc1e44ce5a7e687e3c04e8099` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:10.243426 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e041288-d70a-44ef-856c-c1ad71e04d2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b32b74cc1e44ce5a7e687e3c04e8099)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:10.245399 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e041288-d70a-44ef-856c-c1ad71e04d2a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:10.281113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:10.311174 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8e041288-d70a-44ef-856c-c1ad71e04d2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b32b74cc1e44ce5a7e687e3c04e8099'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:10.324671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8e041288-d70a-44ef-856c-c1ad71e04d2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b32b74cc1e44ce5a7e687e3c04e8099'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:10.333107 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8e041288-d70a-44ef-856c-c1ad71e04d2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b32b74cc1e44ce5a7e687e3c04e8099'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:10.341386 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8e041288-d70a-44ef-856c-c1ad71e04d2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b32b74cc1e44ce5a7e687e3c04e8099'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:10.342205 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8e041288-d70a-44ef-856c-c1ad71e04d2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b32b74cc1e44ce5a7e687e3c04e8099'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:10.351250 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1378/2769] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:25:10 2022] DELETE /identity/v3/users/2b32b74cc1e44ce5a7e687e3c04e8099 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:10.531606 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe686473-ce2f-4da5-97d7-baad0de1c1db service placement] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:10.546908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe686473-ce2f-4da5-97d7-baad0de1c1db service placement] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:10.620527 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0192ffa4-2199-4b9d-8010-5221c3fa1b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:10.621307 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192ffa4-2199-4b9d-8010-5221c3fa1b05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:10.621595 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192ffa4-2199-4b9d-8010-5221c3fa1b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:10.622233 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192ffa4-2199-4b9d-8010-5221c3fa1b05 admin admin] PATH_INFO: `/v3/projects/5411e6d1c7a94aeeaaa8a0a30fb4887b` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:10.623171 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0192ffa4-2199-4b9d-8010-5221c3fa1b05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5411e6d1c7a94aeeaaa8a0a30fb4887b)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:10.625113 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0192ffa4-2199-4b9d-8010-5221c3fa1b05 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:10.684764 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-0192ffa4-2199-4b9d-8010-5221c3fa1b05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5411e6d1c7a94aeeaaa8a0a30fb4887b', 'request_id': 'req-0192ffa4-2199-4b9d-8010-5221c3fa1b05'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:10.691392 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-0192ffa4-2199-4b9d-8010-5221c3fa1b05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5411e6d1c7a94aeeaaa8a0a30fb4887b', 'request_id': 'req-0192ffa4-2199-4b9d-8010-5221c3fa1b05'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:10.703252 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-0192ffa4-2199-4b9d-8010-5221c3fa1b05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5411e6d1c7a94aeeaaa8a0a30fb4887b', 'request_id': 'req-0192ffa4-2199-4b9d-8010-5221c3fa1b05'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:10.717568 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-0192ffa4-2199-4b9d-8010-5221c3fa1b05 admin admin] The token cache is being invalidate because project 5411e6d1c7a94aeeaaa8a0a30fb4887b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:25:10.717869 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-0192ffa4-2199-4b9d-8010-5221c3fa1b05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:10.718560 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-0192ffa4-2199-4b9d-8010-5221c3fa1b05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:10.720113 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1392/2770] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:25:10 2022] DELETE /identity/v3/projects/5411e6d1c7a94aeeaaa8a0a30fb4887b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:25:10.725273 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5313c-e848-47f3-9700-d5046cf3d2a6 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:10.725577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5313c-e848-47f3-9700-d5046cf3d2a6 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:10.725862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5313c-e848-47f3-9700-d5046cf3d2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:10.789508 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-19d5313c-e848-47f3-9700-d5046cf3d2a6 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:10.790314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-19d5313c-e848-47f3-9700-d5046cf3d2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:10.856614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1379/2771] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:25:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:10.865400 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7687b5a-0c5b-4f40-9e95-d9d22dbe25e9 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:10.865694 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7687b5a-0c5b-4f40-9e95-d9d22dbe25e9 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:10.866013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7687b5a-0c5b-4f40-9e95-d9d22dbe25e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:10.884542 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-e7687b5a-0c5b-4f40-9e95-d9d22dbe25e9 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:10.885977 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-e7687b5a-0c5b-4f40-9e95-d9d22dbe25e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:10.895559 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1393/2772] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:25:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:10.899856 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5313c-e848-47f3-9700-d5046cf3d2a6 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:10.910517 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5313c-e848-47f3-9700-d5046cf3d2a6 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:10.915373 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b16c50-5bdf-41cd-b94f-3478b770f932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:10.916040 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b16c50-5bdf-41cd-b94f-3478b770f932 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:10.916325 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b16c50-5bdf-41cd-b94f-3478b770f932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:10.916603 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b16c50-5bdf-41cd-b94f-3478b770f932 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:10.917329 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b16c50-5bdf-41cd-b94f-3478b770f932 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:25:10.917639 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b16c50-5bdf-41cd-b94f-3478b770f932 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:10.919312 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b16c50-5bdf-41cd-b94f-3478b770f932 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:10.931702 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1380/2773] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:25:10.938696 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7687b5a-0c5b-4f40-9e95-d9d22dbe25e9 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:10.948191 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7687b5a-0c5b-4f40-9e95-d9d22dbe25e9 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:10.953793 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28cc00-df3d-486b-bda7-54ccbc0bb13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:10.954485 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28cc00-df3d-486b-bda7-54ccbc0bb13c admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:10.954772 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28cc00-df3d-486b-bda7-54ccbc0bb13c admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:10.955053 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28cc00-df3d-486b-bda7-54ccbc0bb13c admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:10.955752 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28cc00-df3d-486b-bda7-54ccbc0bb13c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:10.957749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28cc00-df3d-486b-bda7-54ccbc0bb13c admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:10.985797 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1394/2774] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:25:10 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:10.989914 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b16c50-5bdf-41cd-b94f-3478b770f932 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:11.001126 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b16c50-5bdf-41cd-b94f-3478b770f932 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:11.060877 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:11.068000 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21532631-4a5c-46bf-9dd0-7bc6ee5872c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:11.069016 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21532631-4a5c-46bf-9dd0-7bc6ee5872c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:11.069484 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21532631-4a5c-46bf-9dd0-7bc6ee5872c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:11.069773 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21532631-4a5c-46bf-9dd0-7bc6ee5872c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:11.070811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21532631-4a5c-46bf-9dd0-7bc6ee5872c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:11.073154 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21532631-4a5c-46bf-9dd0-7bc6ee5872c9 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:11.095645 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1381/2775] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:25:10 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:25:11.100206 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28cc00-df3d-486b-bda7-54ccbc0bb13c admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:11.112113 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28cc00-df3d-486b-bda7-54ccbc0bb13c admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:11.117225 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a322bc2-a1cd-4f03-9bfc-5a26fd57cf8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:11.117902 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a322bc2-a1cd-4f03-9bfc-5a26fd57cf8c admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:11.118223 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a322bc2-a1cd-4f03-9bfc-5a26fd57cf8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:11.118494 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a322bc2-a1cd-4f03-9bfc-5a26fd57cf8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:11.119239 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a322bc2-a1cd-4f03-9bfc-5a26fd57cf8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:11.120848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a322bc2-a1cd-4f03-9bfc-5a26fd57cf8c admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:11.133445 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1395/2776] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:25:11.138795 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21532631-4a5c-46bf-9dd0-7bc6ee5872c9 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:11.151626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21532631-4a5c-46bf-9dd0-7bc6ee5872c9 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:11.158777 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec3d70-307d-41a0-8a87-31ba29dd9e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:11.159162 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec3d70-307d-41a0-8a87-31ba29dd9e90 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:11.159422 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec3d70-307d-41a0-8a87-31ba29dd9e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:11.159704 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec3d70-307d-41a0-8a87-31ba29dd9e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:11.160452 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec3d70-307d-41a0-8a87-31ba29dd9e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:11.163266 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec3d70-307d-41a0-8a87-31ba29dd9e90 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:11.175416 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1382/2777] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:25:11.179496 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a322bc2-a1cd-4f03-9bfc-5a26fd57cf8c admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:11.187265 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:11.194669 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a322bc2-a1cd-4f03-9bfc-5a26fd57cf8c admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:11.199608 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ce770-2b08-49e1-9431-195f2b7b18b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:11.200544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ce770-2b08-49e1-9431-195f2b7b18b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:11.200837 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ce770-2b08-49e1-9431-195f2b7b18b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:11.201197 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ce770-2b08-49e1-9431-195f2b7b18b2 admin admin] PATH_INFO: `/v3/projects/2aba1902188b4aeea6f353c25692a234/users/34b79c4877c948bfb09be2fefb4d71e9/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:11.220655 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776ce770-2b08-49e1-9431-195f2b7b18b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aba1902188b4aeea6f353c25692a234, user_id=34b79c4877c948bfb09be2fefb4d71e9, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:11.223571 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776ce770-2b08-49e1-9431-195f2b7b18b2 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:11.239815 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1396/2778] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:25:11 2022] PUT /identity/v3/projects/2aba1902188b4aeea6f353c25692a234/users/34b79c4877c948bfb09be2fefb4d71e9/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:11.252094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54e32c-5f3e-445c-89a0-002638d900fa None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:11.253108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54e32c-5f3e-445c-89a0-002638d900fa None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:11.253108 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54e32c-5f3e-445c-89a0-002638d900fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:11.283157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0f54e32c-5f3e-445c-89a0-002638d900fa None None] MFA Rules not processed for user `34b79c4877c948bfb09be2fefb4d71e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:11.284406 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0f54e32c-5f3e-445c-89a0-002638d900fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:11.363095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1383/2779] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:25:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:11.375882 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ce770-2b08-49e1-9431-195f2b7b18b2 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:11.377858 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-776ce770-2b08-49e1-9431-195f2b7b18b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:11.477561 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ce770-2b08-49e1-9431-195f2b7b18b2 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:11.483779 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c35679-7538-4b2f-9333-481c394d151c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3645b1a0be34c01ac87c734fcd85ff6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:11.484435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c35679-7538-4b2f-9333-481c394d151c service nova] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:11.484749 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c35679-7538-4b2f-9333-481c394d151c service nova] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:11.485000 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c35679-7538-4b2f-9333-481c394d151c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:11.493378 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c35679-7538-4b2f-9333-481c394d151c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:11.495014 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c35679-7538-4b2f-9333-481c394d151c service nova] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:11.504852 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1397/2780] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:25:18.896223 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54e32c-5f3e-445c-89a0-002638d900fa None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:18.963193 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54e32c-5f3e-445c-89a0-002638d900fa None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:18.969103 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d2e44-b0dd-4112-b2eb-604aa4fd485f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:18.970052 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d2e44-b0dd-4112-b2eb-604aa4fd485f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:18.970336 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d2e44-b0dd-4112-b2eb-604aa4fd485f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:18.970615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d2e44-b0dd-4112-b2eb-604aa4fd485f admin admin] PATH_INFO: `/v3/users/34b79c4877c948bfb09be2fefb4d71e9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:18.971849 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d2e44-b0dd-4112-b2eb-604aa4fd485f admin admin] RBAC: Authorizing `identity:delete_user(user_id=34b79c4877c948bfb09be2fefb4d71e9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:18.973819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d2e44-b0dd-4112-b2eb-604aa4fd485f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:19.027158 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-712d2e44-b0dd-4112-b2eb-604aa4fd485f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34b79c4877c948bfb09be2fefb4d71e9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:19.040761 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-712d2e44-b0dd-4112-b2eb-604aa4fd485f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34b79c4877c948bfb09be2fefb4d71e9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:19.043417 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:19.050822 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-712d2e44-b0dd-4112-b2eb-604aa4fd485f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34b79c4877c948bfb09be2fefb4d71e9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:19.063058 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-712d2e44-b0dd-4112-b2eb-604aa4fd485f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34b79c4877c948bfb09be2fefb4d71e9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:19.064464 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-712d2e44-b0dd-4112-b2eb-604aa4fd485f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34b79c4877c948bfb09be2fefb4d71e9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:19.075415 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1384/2781] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:25:18 2022] DELETE /identity/v3/users/34b79c4877c948bfb09be2fefb4d71e9 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:19.080731 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62ed9f-724a-42f7-b7db-be4c11aceb0f None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:19.081519 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62ed9f-724a-42f7-b7db-be4c11aceb0f None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:19.081710 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62ed9f-724a-42f7-b7db-be4c11aceb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:19.100506 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-0a62ed9f-724a-42f7-b7db-be4c11aceb0f None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:19.101718 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-0a62ed9f-724a-42f7-b7db-be4c11aceb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:19.184282 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1398/2782] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:25:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:19.196709 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d2e44-b0dd-4112-b2eb-604aa4fd485f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:19.198227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-712d2e44-b0dd-4112-b2eb-604aa4fd485f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:19.276958 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d2e44-b0dd-4112-b2eb-604aa4fd485f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:19.280781 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad51e2-b7ce-4e93-b926-c1bc01318af4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:19.281379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad51e2-b7ce-4e93-b926-c1bc01318af4 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:19.281661 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad51e2-b7ce-4e93-b926-c1bc01318af4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:19.281951 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad51e2-b7ce-4e93-b926-c1bc01318af4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:19.291625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad51e2-b7ce-4e93-b926-c1bc01318af4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:19.293061 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad51e2-b7ce-4e93-b926-c1bc01318af4 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:19.302837 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1385/2783] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:19.599367 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62ed9f-724a-42f7-b7db-be4c11aceb0f None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:19.600827 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-0a62ed9f-724a-42f7-b7db-be4c11aceb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:19.613808 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62ed9f-724a-42f7-b7db-be4c11aceb0f None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:19.619082 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9bb2ee-9d9d-48ce-9a43-af0ba467e6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:19.619966 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bb2ee-9d9d-48ce-9a43-af0ba467e6d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:19.620292 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bb2ee-9d9d-48ce-9a43-af0ba467e6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:19.620566 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bb2ee-9d9d-48ce-9a43-af0ba467e6d8 admin admin] PATH_INFO: `/v3/projects/2aba1902188b4aeea6f353c25692a234` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:19.621890 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9bb2ee-9d9d-48ce-9a43-af0ba467e6d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aba1902188b4aeea6f353c25692a234)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:19.625392 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9bb2ee-9d9d-48ce-9a43-af0ba467e6d8 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:19.683883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-1c9bb2ee-9d9d-48ce-9a43-af0ba467e6d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aba1902188b4aeea6f353c25692a234', 'request_id': 'req-1c9bb2ee-9d9d-48ce-9a43-af0ba467e6d8'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:19.688772 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:19.691222 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-1c9bb2ee-9d9d-48ce-9a43-af0ba467e6d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aba1902188b4aeea6f353c25692a234', 'request_id': 'req-1c9bb2ee-9d9d-48ce-9a43-af0ba467e6d8'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:19.700070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-1c9bb2ee-9d9d-48ce-9a43-af0ba467e6d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aba1902188b4aeea6f353c25692a234', 'request_id': 'req-1c9bb2ee-9d9d-48ce-9a43-af0ba467e6d8'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:19.713227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-1c9bb2ee-9d9d-48ce-9a43-af0ba467e6d8 admin admin] The token cache is being invalidate because project 2aba1902188b4aeea6f353c25692a234 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:25:19.713510 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-1c9bb2ee-9d9d-48ce-9a43-af0ba467e6d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:19.714248 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-1c9bb2ee-9d9d-48ce-9a43-af0ba467e6d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:19.715917 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1399/2784] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:25:19 2022] DELETE /identity/v3/projects/2aba1902188b4aeea6f353c25692a234 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:19.721074 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e48a42-e3c2-4c16-8cc3-07dc9e327ba0 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:19.721364 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e48a42-e3c2-4c16-8cc3-07dc9e327ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:19.721811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e48a42-e3c2-4c16-8cc3-07dc9e327ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:19.742418 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d2e48a42-e3c2-4c16-8cc3-07dc9e327ba0 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:19.743785 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d2e48a42-e3c2-4c16-8cc3-07dc9e327ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:19.820929 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1386/2785] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:25:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:25:19.829919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc92c5-1817-4019-8074-dc9c02ddde95 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:19.830381 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc92c5-1817-4019-8074-dc9c02ddde95 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:19.831078 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc92c5-1817-4019-8074-dc9c02ddde95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:19.851552 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-02fc92c5-1817-4019-8074-dc9c02ddde95 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:19.852795 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-02fc92c5-1817-4019-8074-dc9c02ddde95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:19.863172 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1400/2786] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:25:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:19.867622 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e48a42-e3c2-4c16-8cc3-07dc9e327ba0 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:19.878828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e48a42-e3c2-4c16-8cc3-07dc9e327ba0 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:19.884401 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4538ab-05b5-4866-b8ea-23a60b3e5cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:19.885327 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4538ab-05b5-4866-b8ea-23a60b3e5cfe admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:19.885603 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4538ab-05b5-4866-b8ea-23a60b3e5cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:19.885868 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4538ab-05b5-4866-b8ea-23a60b3e5cfe admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:19.886637 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4538ab-05b5-4866-b8ea-23a60b3e5cfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:25:19.886943 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4538ab-05b5-4866-b8ea-23a60b3e5cfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:19.888783 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4538ab-05b5-4866-b8ea-23a60b3e5cfe admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:19.902106 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1387/2787] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:25:19.907376 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc92c5-1817-4019-8074-dc9c02ddde95 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:19.917821 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc92c5-1817-4019-8074-dc9c02ddde95 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:19.923886 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ff32c-a541-44b1-a907-3bea94ceff40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:19.924580 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ff32c-a541-44b1-a907-3bea94ceff40 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:19.924859 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ff32c-a541-44b1-a907-3bea94ceff40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:19.925129 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ff32c-a541-44b1-a907-3bea94ceff40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:19.925833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ff32c-a541-44b1-a907-3bea94ceff40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:19.927800 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ff32c-a541-44b1-a907-3bea94ceff40 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:19.959723 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1401/2788] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:25:19 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:19.964129 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4538ab-05b5-4866-b8ea-23a60b3e5cfe admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:19.976346 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4538ab-05b5-4866-b8ea-23a60b3e5cfe admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:20.040427 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5288d-fc3b-48f9-bb9c-c80dd48470ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:20.041050 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5288d-fc3b-48f9-bb9c-c80dd48470ae admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:20.041461 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5288d-fc3b-48f9-bb9c-c80dd48470ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:20.041695 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5288d-fc3b-48f9-bb9c-c80dd48470ae admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:20.042465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc5288d-fc3b-48f9-bb9c-c80dd48470ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:20.045028 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc5288d-fc3b-48f9-bb9c-c80dd48470ae admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:20.055075 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:20.073321 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1388/2789] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:25:19 2022] POST /identity/v3/users => generated 283 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:20.078790 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ff32c-a541-44b1-a907-3bea94ceff40 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:20.089809 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ff32c-a541-44b1-a907-3bea94ceff40 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:20.095630 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990d8e8-f5e2-429b-9d84-8e1bdae9ecb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:20.096432 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990d8e8-f5e2-429b-9d84-8e1bdae9ecb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:20.096716 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990d8e8-f5e2-429b-9d84-8e1bdae9ecb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:20.096999 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990d8e8-f5e2-429b-9d84-8e1bdae9ecb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:20.097733 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7990d8e8-f5e2-429b-9d84-8e1bdae9ecb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:20.099407 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7990d8e8-f5e2-429b-9d84-8e1bdae9ecb2 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:20.112442 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1402/2790] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:25:20.116524 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5288d-fc3b-48f9-bb9c-c80dd48470ae admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:20.127883 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5288d-fc3b-48f9-bb9c-c80dd48470ae admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:20.132959 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01292819-0f8c-4f1c-986d-bc38c8d5f149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:20.133703 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01292819-0f8c-4f1c-986d-bc38c8d5f149 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:20.133983 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01292819-0f8c-4f1c-986d-bc38c8d5f149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:20.134268 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01292819-0f8c-4f1c-986d-bc38c8d5f149 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:20.135274 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01292819-0f8c-4f1c-986d-bc38c8d5f149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:20.136887 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01292819-0f8c-4f1c-986d-bc38c8d5f149 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:20.148040 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1389/2791] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:25:20.152395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990d8e8-f5e2-429b-9d84-8e1bdae9ecb2 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:20.164185 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990d8e8-f5e2-429b-9d84-8e1bdae9ecb2 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:20.169377 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f140e-cf74-4fdb-b32c-b2eeabe586fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:20.170851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f140e-cf74-4fdb-b32c-b2eeabe586fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:20.171191 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f140e-cf74-4fdb-b32c-b2eeabe586fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:20.171478 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f140e-cf74-4fdb-b32c-b2eeabe586fb admin admin] PATH_INFO: `/v3/projects/d8c347c4de804ae09c2f3540dbec28d6/users/4b73695f8c334cfdaf127c5afbdfc2b1/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:20.189750 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f140e-cf74-4fdb-b32c-b2eeabe586fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8c347c4de804ae09c2f3540dbec28d6, user_id=4b73695f8c334cfdaf127c5afbdfc2b1, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:20.192621 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f140e-cf74-4fdb-b32c-b2eeabe586fb admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:20.206262 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1403/2792] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:25:20 2022] PUT /identity/v3/projects/d8c347c4de804ae09c2f3540dbec28d6/users/4b73695f8c334cfdaf127c5afbdfc2b1/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:20.219233 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18a1bb-b507-42f4-8c78-4e53b3bd936b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:20.219233 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18a1bb-b507-42f4-8c78-4e53b3bd936b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:20.219233 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18a1bb-b507-42f4-8c78-4e53b3bd936b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:20.252049 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-5b18a1bb-b507-42f4-8c78-4e53b3bd936b None None] MFA Rules not processed for user `4b73695f8c334cfdaf127c5afbdfc2b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:20.253349 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5b18a1bb-b507-42f4-8c78-4e53b3bd936b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:20.338052 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1390/2793] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:25:20 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:25:20.343386 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f140e-cf74-4fdb-b32c-b2eeabe586fb admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:20.407958 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f140e-cf74-4fdb-b32c-b2eeabe586fb admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:20.412857 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb32e2-ee17-4b00-8b81-a3d63a485c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:20.413529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb32e2-ee17-4b00-8b81-a3d63a485c0a admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:20.413815 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb32e2-ee17-4b00-8b81-a3d63a485c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:20.414128 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb32e2-ee17-4b00-8b81-a3d63a485c0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:20.414923 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb32e2-ee17-4b00-8b81-a3d63a485c0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:20.416863 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb32e2-ee17-4b00-8b81-a3d63a485c0a admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:20.449246 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1404/2794] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:25:20 2022] POST /identity/v3/projects => generated 363 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:20.454305 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b18a1bb-b507-42f4-8c78-4e53b3bd936b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:20.467819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b18a1bb-b507-42f4-8c78-4e53b3bd936b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:20.528237 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d33623-6941-4e13-9b88-a01f684e288b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:20.528925 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d33623-6941-4e13-9b88-a01f684e288b admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:20.529212 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d33623-6941-4e13-9b88-a01f684e288b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:20.529489 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d33623-6941-4e13-9b88-a01f684e288b admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:20.530224 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d33623-6941-4e13-9b88-a01f684e288b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:20.532165 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d33623-6941-4e13-9b88-a01f684e288b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:20.553899 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1391/2795] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:25:20 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:20.557867 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb32e2-ee17-4b00-8b81-a3d63a485c0a admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:20.570549 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb32e2-ee17-4b00-8b81-a3d63a485c0a admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:20.575351 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e89b8-fc30-4c24-983f-e6ecfc2c98e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:20.576061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e89b8-fc30-4c24-983f-e6ecfc2c98e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:20.576348 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e89b8-fc30-4c24-983f-e6ecfc2c98e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:20.576848 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e89b8-fc30-4c24-983f-e6ecfc2c98e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:20.577586 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e89b8-fc30-4c24-983f-e6ecfc2c98e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:20.579257 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e89b8-fc30-4c24-983f-e6ecfc2c98e8 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:20.590676 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1405/2796] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:25:20.595151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d33623-6941-4e13-9b88-a01f684e288b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:20.605794 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d33623-6941-4e13-9b88-a01f684e288b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:20.612258 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be19ae96-e532-4a3d-8925-8216f2385de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:20.613166 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19ae96-e532-4a3d-8925-8216f2385de3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:20.613483 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19ae96-e532-4a3d-8925-8216f2385de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:20.613760 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19ae96-e532-4a3d-8925-8216f2385de3 admin admin] PATH_INFO: `/v3/projects/3c37f426c6d74e09adb4c1c174ffc039/users/e609df2506944bfe9b4c61352191f88c/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:20.632303 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be19ae96-e532-4a3d-8925-8216f2385de3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c37f426c6d74e09adb4c1c174ffc039, user_id=e609df2506944bfe9b4c61352191f88c, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:20.635656 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be19ae96-e532-4a3d-8925-8216f2385de3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:20.651499 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1392/2797] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:25:20 2022] PUT /identity/v3/projects/3c37f426c6d74e09adb4c1c174ffc039/users/e609df2506944bfe9b4c61352191f88c/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:20.655900 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e89b8-fc30-4c24-983f-e6ecfc2c98e8 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:20.666466 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e89b8-fc30-4c24-983f-e6ecfc2c98e8 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:20.721875 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:20.729124 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1a2fe-84b5-4f38-a192-834b0521d2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:20.729805 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1a2fe-84b5-4f38-a192-834b0521d2a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:20.730120 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1a2fe-84b5-4f38-a192-834b0521d2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:20.730405 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1a2fe-84b5-4f38-a192-834b0521d2a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:20.731171 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1a2fe-84b5-4f38-a192-834b0521d2a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:20.733529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1a2fe-84b5-4f38-a192-834b0521d2a0 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:20.745591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1406/2798] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:25:20.749780 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be19ae96-e532-4a3d-8925-8216f2385de3 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:20.760708 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be19ae96-e532-4a3d-8925-8216f2385de3 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:20.765591 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d8df9-b00f-475c-9791-ee8bf87ae073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:20.766538 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d8df9-b00f-475c-9791-ee8bf87ae073 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:20.766826 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d8df9-b00f-475c-9791-ee8bf87ae073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:20.767104 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d8df9-b00f-475c-9791-ee8bf87ae073 admin admin] PATH_INFO: `/v3/projects/3c37f426c6d74e09adb4c1c174ffc039/users/e609df2506944bfe9b4c61352191f88c/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:20.769121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d8df9-b00f-475c-9791-ee8bf87ae073 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c37f426c6d74e09adb4c1c174ffc039, user_id=e609df2506944bfe9b4c61352191f88c, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:20.771854 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d8df9-b00f-475c-9791-ee8bf87ae073 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:20.783832 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1393/2799] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:25:20 2022] PUT /identity/v3/projects/3c37f426c6d74e09adb4c1c174ffc039/users/e609df2506944bfe9b4c61352191f88c/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:20.795123 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a242b-43c7-480e-ba1f-5724453e84e1 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:20.795422 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a242b-43c7-480e-ba1f-5724453e84e1 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:20.795697 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a242b-43c7-480e-ba1f-5724453e84e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:20.825516 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-fd8a242b-43c7-480e-ba1f-5724453e84e1 None None] MFA Rules not processed for user `e609df2506944bfe9b4c61352191f88c`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:20.826860 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-fd8a242b-43c7-480e-ba1f-5724453e84e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:20.909100 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1407/2800] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:25:20 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:20.921276 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d8df9-b00f-475c-9791-ee8bf87ae073 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:20.923073 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1f5d8df9-b00f-475c-9791-ee8bf87ae073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:20.954369 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:20.989634 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d8df9-b00f-475c-9791-ee8bf87ae073 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:20.993483 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56961fa5-1353-4481-96e0-b7cd2b02f48e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:20.994653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56961fa5-1353-4481-96e0-b7cd2b02f48e service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:20.994653 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56961fa5-1353-4481-96e0-b7cd2b02f48e service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:20.995131 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56961fa5-1353-4481-96e0-b7cd2b02f48e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:21.004780 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56961fa5-1353-4481-96e0-b7cd2b02f48e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:21.006259 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56961fa5-1353-4481-96e0-b7cd2b02f48e service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:21.016737 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1394/2801] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:20 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:21.469777 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a242b-43c7-480e-ba1f-5724453e84e1 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:21.537515 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a242b-43c7-480e-ba1f-5724453e84e1 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:21.542233 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f721284-0292-4e17-8a9b-d9e6f278cf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:21.543106 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f721284-0292-4e17-8a9b-d9e6f278cf8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:21.543391 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f721284-0292-4e17-8a9b-d9e6f278cf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:21.543669 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f721284-0292-4e17-8a9b-d9e6f278cf8f admin admin] PATH_INFO: `/v3/users/4b73695f8c334cfdaf127c5afbdfc2b1` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:21.545267 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f721284-0292-4e17-8a9b-d9e6f278cf8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b73695f8c334cfdaf127c5afbdfc2b1)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:21.547274 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f721284-0292-4e17-8a9b-d9e6f278cf8f admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:21.600997 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6f721284-0292-4e17-8a9b-d9e6f278cf8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b73695f8c334cfdaf127c5afbdfc2b1'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:21.608731 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6f721284-0292-4e17-8a9b-d9e6f278cf8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b73695f8c334cfdaf127c5afbdfc2b1'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:21.621706 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6f721284-0292-4e17-8a9b-d9e6f278cf8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b73695f8c334cfdaf127c5afbdfc2b1'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:21.623445 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6f721284-0292-4e17-8a9b-d9e6f278cf8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b73695f8c334cfdaf127c5afbdfc2b1'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:21.626191 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:21.632172 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6f721284-0292-4e17-8a9b-d9e6f278cf8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b73695f8c334cfdaf127c5afbdfc2b1'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:21.641142 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1408/2802] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:25:21 2022] DELETE /identity/v3/users/4b73695f8c334cfdaf127c5afbdfc2b1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:21.646145 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f198e78b-f69b-49f3-ae95-dd2771898d39 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:21.646595 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f198e78b-f69b-49f3-ae95-dd2771898d39 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:21.646912 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f198e78b-f69b-49f3-ae95-dd2771898d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:21.666654 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f198e78b-f69b-49f3-ae95-dd2771898d39 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:21.667938 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f198e78b-f69b-49f3-ae95-dd2771898d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:21.736472 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1395/2803] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:21.746699 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f721284-0292-4e17-8a9b-d9e6f278cf8f admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:21.748912 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-6f721284-0292-4e17-8a9b-d9e6f278cf8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:21.817768 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f721284-0292-4e17-8a9b-d9e6f278cf8f admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:21.822296 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f2f5e-c3ce-4187-ade0-a7dc093dd6c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:21.822957 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f2f5e-c3ce-4187-ade0-a7dc093dd6c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:21.823240 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f2f5e-c3ce-4187-ade0-a7dc093dd6c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:21.823641 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f2f5e-c3ce-4187-ade0-a7dc093dd6c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:21.834935 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15f2f5e-c3ce-4187-ade0-a7dc093dd6c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:21.836460 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15f2f5e-c3ce-4187-ade0-a7dc093dd6c2 service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:21.847724 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1409/2804] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:22.135070 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f198e78b-f69b-49f3-ae95-dd2771898d39 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:22.136551 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f198e78b-f69b-49f3-ae95-dd2771898d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:22.149058 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f198e78b-f69b-49f3-ae95-dd2771898d39 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:22.155284 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a5765-8fc5-48c4-ab21-a50990b319b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:22.156671 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a5765-8fc5-48c4-ab21-a50990b319b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:22.157193 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a5765-8fc5-48c4-ab21-a50990b319b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:22.157486 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a5765-8fc5-48c4-ab21-a50990b319b1 admin admin] PATH_INFO: `/v3/projects/d8c347c4de804ae09c2f3540dbec28d6` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:22.159198 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7a5765-8fc5-48c4-ab21-a50990b319b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8c347c4de804ae09c2f3540dbec28d6)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:22.163293 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7a5765-8fc5-48c4-ab21-a50990b319b1 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:22.221656 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ce7a5765-8fc5-48c4-ab21-a50990b319b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8c347c4de804ae09c2f3540dbec28d6', 'request_id': 'req-ce7a5765-8fc5-48c4-ab21-a50990b319b1'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:22.227660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ce7a5765-8fc5-48c4-ab21-a50990b319b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8c347c4de804ae09c2f3540dbec28d6', 'request_id': 'req-ce7a5765-8fc5-48c4-ab21-a50990b319b1'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:22.236422 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ce7a5765-8fc5-48c4-ab21-a50990b319b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8c347c4de804ae09c2f3540dbec28d6', 'request_id': 'req-ce7a5765-8fc5-48c4-ab21-a50990b319b1'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:22.250615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ce7a5765-8fc5-48c4-ab21-a50990b319b1 admin admin] The token cache is being invalidate because project d8c347c4de804ae09c2f3540dbec28d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:25:22.250812 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ce7a5765-8fc5-48c4-ab21-a50990b319b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:22.251530 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ce7a5765-8fc5-48c4-ab21-a50990b319b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:22.253442 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1396/2805] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:25:22 2022] DELETE /identity/v3/projects/d8c347c4de804ae09c2f3540dbec28d6 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:22.257636 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f2f5e-c3ce-4187-ade0-a7dc093dd6c2 service neutron] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:22.270041 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f2f5e-c3ce-4187-ade0-a7dc093dd6c2 service neutron] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:22.340963 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdba1f9-7fbd-4ea8-8eb5-b3a4c9e1ab83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:22.342451 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdba1f9-7fbd-4ea8-8eb5-b3a4c9e1ab83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:22.342741 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdba1f9-7fbd-4ea8-8eb5-b3a4c9e1ab83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:22.343019 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdba1f9-7fbd-4ea8-8eb5-b3a4c9e1ab83 admin admin] PATH_INFO: `/v3/users/e609df2506944bfe9b4c61352191f88c` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:22.344280 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdba1f9-7fbd-4ea8-8eb5-b3a4c9e1ab83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e609df2506944bfe9b4c61352191f88c)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:22.346265 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdba1f9-7fbd-4ea8-8eb5-b3a4c9e1ab83 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:22.401906 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-ffdba1f9-7fbd-4ea8-8eb5-b3a4c9e1ab83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e609df2506944bfe9b4c61352191f88c'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:22.409084 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-ffdba1f9-7fbd-4ea8-8eb5-b3a4c9e1ab83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e609df2506944bfe9b4c61352191f88c'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:22.422335 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-ffdba1f9-7fbd-4ea8-8eb5-b3a4c9e1ab83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e609df2506944bfe9b4c61352191f88c'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:22.423291 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-ffdba1f9-7fbd-4ea8-8eb5-b3a4c9e1ab83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e609df2506944bfe9b4c61352191f88c'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:22.432303 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-ffdba1f9-7fbd-4ea8-8eb5-b3a4c9e1ab83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e609df2506944bfe9b4c61352191f88c'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:22.440064 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1410/2806] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:25:22 2022] DELETE /identity/v3/users/e609df2506944bfe9b4c61352191f88c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:22.732455 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a5765-8fc5-48c4-ab21-a50990b319b1 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:22.734552 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ce7a5765-8fc5-48c4-ab21-a50990b319b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:22.749494 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a5765-8fc5-48c4-ab21-a50990b319b1 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:22.807061 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:22.819082 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad590e8b-d0b6-4e02-9854-9194867ba551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:22.819975 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad590e8b-d0b6-4e02-9854-9194867ba551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:22.820256 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad590e8b-d0b6-4e02-9854-9194867ba551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:22.820526 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad590e8b-d0b6-4e02-9854-9194867ba551 admin admin] PATH_INFO: `/v3/projects/3c37f426c6d74e09adb4c1c174ffc039` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:22.821815 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad590e8b-d0b6-4e02-9854-9194867ba551 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c37f426c6d74e09adb4c1c174ffc039)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:22.823684 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad590e8b-d0b6-4e02-9854-9194867ba551 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:22.874204 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ad590e8b-d0b6-4e02-9854-9194867ba551 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c37f426c6d74e09adb4c1c174ffc039', 'request_id': 'req-ad590e8b-d0b6-4e02-9854-9194867ba551'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:22.879914 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ad590e8b-d0b6-4e02-9854-9194867ba551 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c37f426c6d74e09adb4c1c174ffc039', 'request_id': 'req-ad590e8b-d0b6-4e02-9854-9194867ba551'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:22.888895 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ad590e8b-d0b6-4e02-9854-9194867ba551 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c37f426c6d74e09adb4c1c174ffc039', 'request_id': 'req-ad590e8b-d0b6-4e02-9854-9194867ba551'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:22.902475 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ad590e8b-d0b6-4e02-9854-9194867ba551 admin admin] The token cache is being invalidate because project 3c37f426c6d74e09adb4c1c174ffc039 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:25:22.906290 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ad590e8b-d0b6-4e02-9854-9194867ba551 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:22.906290 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ad590e8b-d0b6-4e02-9854-9194867ba551 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:22.906290 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1397/2807] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:25:22 2022] DELETE /identity/v3/projects/3c37f426c6d74e09adb4c1c174ffc039 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:22.911795 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22beeaf-130b-4a94-9f63-30dd045657cd None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:22.912407 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22beeaf-130b-4a94-9f63-30dd045657cd None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:22.912678 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22beeaf-130b-4a94-9f63-30dd045657cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:22.934238 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-d22beeaf-130b-4a94-9f63-30dd045657cd None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:22.936642 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-d22beeaf-130b-4a94-9f63-30dd045657cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:22.946699 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:23.003234 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1411/2808] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:23.011490 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba700c01-eab1-41dc-80cb-c15b59f08aed None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:23.011787 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba700c01-eab1-41dc-80cb-c15b59f08aed None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:23.012072 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba700c01-eab1-41dc-80cb-c15b59f08aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:23.031818 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ba700c01-eab1-41dc-80cb-c15b59f08aed None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:23.033051 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ba700c01-eab1-41dc-80cb-c15b59f08aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:23.041362 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1398/2809] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:23.046489 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22beeaf-130b-4a94-9f63-30dd045657cd None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:23.058245 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22beeaf-130b-4a94-9f63-30dd045657cd None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:23.063294 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b9cc19-2d5e-4546-a59e-a8b0f838afbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:23.063966 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b9cc19-2d5e-4546-a59e-a8b0f838afbd admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:23.064251 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b9cc19-2d5e-4546-a59e-a8b0f838afbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:23.064538 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b9cc19-2d5e-4546-a59e-a8b0f838afbd admin admin] PATH_INFO: `/v3/domains` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:23.065269 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b9cc19-2d5e-4546-a59e-a8b0f838afbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:25:23.065581 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b9cc19-2d5e-4546-a59e-a8b0f838afbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:23.067254 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b9cc19-2d5e-4546-a59e-a8b0f838afbd admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:23.079685 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1412/2810] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:25:23.084079 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba700c01-eab1-41dc-80cb-c15b59f08aed None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:23.095040 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba700c01-eab1-41dc-80cb-c15b59f08aed None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:23.100023 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2eb74-7e43-4783-b145-29fe9047a5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:23.100690 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2eb74-7e43-4783-b145-29fe9047a5d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:23.100968 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2eb74-7e43-4783-b145-29fe9047a5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:23.101241 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2eb74-7e43-4783-b145-29fe9047a5d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:23.101952 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f2eb74-7e43-4783-b145-29fe9047a5d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:23.104006 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f2eb74-7e43-4783-b145-29fe9047a5d4 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:23.139362 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1399/2811] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:25:23 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:23.144004 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b9cc19-2d5e-4546-a59e-a8b0f838afbd admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:23.154816 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b9cc19-2d5e-4546-a59e-a8b0f838afbd admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:23.215811 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95096a67-718e-49f4-93c3-c5c8a6047358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:23.217402 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95096a67-718e-49f4-93c3-c5c8a6047358 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:23.217402 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95096a67-718e-49f4-93c3-c5c8a6047358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:23.217685 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95096a67-718e-49f4-93c3-c5c8a6047358 admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:23.218379 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95096a67-718e-49f4-93c3-c5c8a6047358 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:23.220391 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95096a67-718e-49f4-93c3-c5c8a6047358 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:23.243487 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1413/2812] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:25:23 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:23.248580 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2eb74-7e43-4783-b145-29fe9047a5d4 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:23.259470 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2eb74-7e43-4783-b145-29fe9047a5d4 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:23.265367 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba58ce1-623d-4897-b221-205e2b452c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:23.266074 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba58ce1-623d-4897-b221-205e2b452c89 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:23.266418 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba58ce1-623d-4897-b221-205e2b452c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:23.266956 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba58ce1-623d-4897-b221-205e2b452c89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:23.267955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba58ce1-623d-4897-b221-205e2b452c89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:23.269682 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba58ce1-623d-4897-b221-205e2b452c89 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:23.281096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1400/2813] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:25:23.285690 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95096a67-718e-49f4-93c3-c5c8a6047358 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:23.297218 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95096a67-718e-49f4-93c3-c5c8a6047358 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:23.302165 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce332351-89f3-42f1-9d3f-bc6e7e79bce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:23.302833 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce332351-89f3-42f1-9d3f-bc6e7e79bce9 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:23.303120 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce332351-89f3-42f1-9d3f-bc6e7e79bce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:23.303397 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce332351-89f3-42f1-9d3f-bc6e7e79bce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:23.304126 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce332351-89f3-42f1-9d3f-bc6e7e79bce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:23.305757 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce332351-89f3-42f1-9d3f-bc6e7e79bce9 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:23.317544 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1414/2814] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:25:23.325504 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba58ce1-623d-4897-b221-205e2b452c89 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:23.337115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba58ce1-623d-4897-b221-205e2b452c89 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:23.342810 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0c757-a386-45e5-b95d-8ca60810c2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:23.343718 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0c757-a386-45e5-b95d-8ca60810c2b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:23.344047 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0c757-a386-45e5-b95d-8ca60810c2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:23.344331 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0c757-a386-45e5-b95d-8ca60810c2b2 admin admin] PATH_INFO: `/v3/projects/45ef41dbf1d74072a0147fd501819e5a/users/521f5c381cf44cfeb0e7e2ea6271d404/roles/3ce570490865493380e6941d004136bd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:23.361240 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b0c757-a386-45e5-b95d-8ca60810c2b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45ef41dbf1d74072a0147fd501819e5a, user_id=521f5c381cf44cfeb0e7e2ea6271d404, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:23.364169 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b0c757-a386-45e5-b95d-8ca60810c2b2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:23.380446 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1401/2815] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:25:23 2022] PUT /identity/v3/projects/45ef41dbf1d74072a0147fd501819e5a/users/521f5c381cf44cfeb0e7e2ea6271d404/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:23.392517 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b709508-2d15-4378-91fe-d60e788fa161 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:23.392838 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b709508-2d15-4378-91fe-d60e788fa161 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:23.393118 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b709508-2d15-4378-91fe-d60e788fa161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:23.420531 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-3b709508-2d15-4378-91fe-d60e788fa161 None None] MFA Rules not processed for user `521f5c381cf44cfeb0e7e2ea6271d404`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:23.421745 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-3b709508-2d15-4378-91fe-d60e788fa161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:23.489870 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1415/2816] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:25:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:23.494392 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0c757-a386-45e5-b95d-8ca60810c2b2 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:23.563060 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0c757-a386-45e5-b95d-8ca60810c2b2 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:23.568415 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0a828-1c6e-4b3b-8114-9666ac8a1087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:23.569017 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0a828-1c6e-4b3b-8114-9666ac8a1087 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:23.569295 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0a828-1c6e-4b3b-8114-9666ac8a1087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:23.569571 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0a828-1c6e-4b3b-8114-9666ac8a1087 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:23.570301 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb0a828-1c6e-4b3b-8114-9666ac8a1087 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:23.572201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb0a828-1c6e-4b3b-8114-9666ac8a1087 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:23.605191 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1402/2817] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:25:23 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:23.610279 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b709508-2d15-4378-91fe-d60e788fa161 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:23.622386 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b709508-2d15-4378-91fe-d60e788fa161 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:23.631733 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:23.688933 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec9205-f5fb-4e20-82ab-2d12160df1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:23.689632 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec9205-f5fb-4e20-82ab-2d12160df1df admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:23.689904 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec9205-f5fb-4e20-82ab-2d12160df1df admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:23.690291 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec9205-f5fb-4e20-82ab-2d12160df1df admin admin] PATH_INFO: `/v3/users` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:23.690974 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec9205-f5fb-4e20-82ab-2d12160df1df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:23.693007 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec9205-f5fb-4e20-82ab-2d12160df1df admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:23.715283 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1416/2818] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:25:23 2022] POST /identity/v3/users => generated 299 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:23.719732 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0a828-1c6e-4b3b-8114-9666ac8a1087 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:23.724939 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:23.731704 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0a828-1c6e-4b3b-8114-9666ac8a1087 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:23.739027 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91357a0b-df23-4ced-a929-daa665498a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:23.739796 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91357a0b-df23-4ced-a929-daa665498a54 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:23.740197 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91357a0b-df23-4ced-a929-daa665498a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:23.740487 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91357a0b-df23-4ced-a929-daa665498a54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:23.741220 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91357a0b-df23-4ced-a929-daa665498a54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:23.742878 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91357a0b-df23-4ced-a929-daa665498a54 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:23.759932 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1403/2819] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:25:23.764807 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec9205-f5fb-4e20-82ab-2d12160df1df admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:23.775635 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec9205-f5fb-4e20-82ab-2d12160df1df admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:23.780891 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add946f0-1fa2-4e46-8748-bf6de15092ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:23.781824 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add946f0-1fa2-4e46-8748-bf6de15092ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:23.782992 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add946f0-1fa2-4e46-8748-bf6de15092ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:23.782992 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add946f0-1fa2-4e46-8748-bf6de15092ad admin admin] PATH_INFO: `/v3/projects/4b4312f5d3934e3ba708c5e25c77a30e/users/539992efa88148a49ff7226414cc6994/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:23.800287 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add946f0-1fa2-4e46-8748-bf6de15092ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b4312f5d3934e3ba708c5e25c77a30e, user_id=539992efa88148a49ff7226414cc6994, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:23.803044 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add946f0-1fa2-4e46-8748-bf6de15092ad admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:23.817691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1417/2820] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:25:23 2022] PUT /identity/v3/projects/4b4312f5d3934e3ba708c5e25c77a30e/users/539992efa88148a49ff7226414cc6994/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:23.822638 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91357a0b-df23-4ced-a929-daa665498a54 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:23.833407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91357a0b-df23-4ced-a929-daa665498a54 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:23.893986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b61a4b-85d4-4382-b6a0-d4a0c4a70a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:23.894667 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b61a4b-85d4-4382-b6a0-d4a0c4a70a8d admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:23.894949 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b61a4b-85d4-4382-b6a0-d4a0c4a70a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:23.895220 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b61a4b-85d4-4382-b6a0-d4a0c4a70a8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:23.895946 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b61a4b-85d4-4382-b6a0-d4a0c4a70a8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:23.897529 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b61a4b-85d4-4382-b6a0-d4a0c4a70a8d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:23.908625 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1404/2821] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:25:23.913209 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add946f0-1fa2-4e46-8748-bf6de15092ad admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:23.924402 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add946f0-1fa2-4e46-8748-bf6de15092ad admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:23.929322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6cb14-1dd8-41a7-bcda-afa5b8c7b114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:23.930266 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6cb14-1dd8-41a7-bcda-afa5b8c7b114 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:23.930614 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6cb14-1dd8-41a7-bcda-afa5b8c7b114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:23.930852 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6cb14-1dd8-41a7-bcda-afa5b8c7b114 admin admin] PATH_INFO: `/v3/projects/4b4312f5d3934e3ba708c5e25c77a30e/users/539992efa88148a49ff7226414cc6994/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:23.932880 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6cb14-1dd8-41a7-bcda-afa5b8c7b114 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b4312f5d3934e3ba708c5e25c77a30e, user_id=539992efa88148a49ff7226414cc6994, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:23.936247 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6cb14-1dd8-41a7-bcda-afa5b8c7b114 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:23.947529 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1418/2822] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:25:23 2022] PUT /identity/v3/projects/4b4312f5d3934e3ba708c5e25c77a30e/users/539992efa88148a49ff7226414cc6994/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:23.958828 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee4b2a-975e-46fd-9161-781936520baf None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:23.959120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee4b2a-975e-46fd-9161-781936520baf None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:23.959401 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee4b2a-975e-46fd-9161-781936520baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:23.987143 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e7ee4b2a-975e-46fd-9161-781936520baf None None] MFA Rules not processed for user `539992efa88148a49ff7226414cc6994`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:23.988371 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e7ee4b2a-975e-46fd-9161-781936520baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:24.071200 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1405/2823] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:25:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:24.082617 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6cb14-1dd8-41a7-bcda-afa5b8c7b114 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:24.084410 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-8bb6cb14-1dd8-41a7-bcda-afa5b8c7b114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:24.143630 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6cb14-1dd8-41a7-bcda-afa5b8c7b114 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:24.147373 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd439235-52fc-446e-81cd-fe9bf0c532b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:24.148024 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd439235-52fc-446e-81cd-fe9bf0c532b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:24.148308 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd439235-52fc-446e-81cd-fe9bf0c532b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:24.148587 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd439235-52fc-446e-81cd-fe9bf0c532b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:24.160296 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd439235-52fc-446e-81cd-fe9bf0c532b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:24.162441 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd439235-52fc-446e-81cd-fe9bf0c532b5 service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:24.172862 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1419/2824] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:24 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:24.200678 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee4b2a-975e-46fd-9161-781936520baf None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:24.266246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee4b2a-975e-46fd-9161-781936520baf None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:24.271054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b774c1-c893-427f-b09c-0577e4e392e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:24.271942 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b774c1-c893-427f-b09c-0577e4e392e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:24.272233 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b774c1-c893-427f-b09c-0577e4e392e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:24.272514 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b774c1-c893-427f-b09c-0577e4e392e2 admin admin] PATH_INFO: `/v3/users/521f5c381cf44cfeb0e7e2ea6271d404` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:24.273753 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b774c1-c893-427f-b09c-0577e4e392e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=521f5c381cf44cfeb0e7e2ea6271d404)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:24.275726 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b774c1-c893-427f-b09c-0577e4e392e2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:24.300121 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:24.346296 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e3b774c1-c893-427f-b09c-0577e4e392e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '521f5c381cf44cfeb0e7e2ea6271d404'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:24.359327 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e3b774c1-c893-427f-b09c-0577e4e392e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '521f5c381cf44cfeb0e7e2ea6271d404'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:24.367561 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e3b774c1-c893-427f-b09c-0577e4e392e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '521f5c381cf44cfeb0e7e2ea6271d404'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:24.375498 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e3b774c1-c893-427f-b09c-0577e4e392e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '521f5c381cf44cfeb0e7e2ea6271d404'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:24.376170 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e3b774c1-c893-427f-b09c-0577e4e392e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '521f5c381cf44cfeb0e7e2ea6271d404'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:24.383006 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1406/2825] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:25:24 2022] DELETE /identity/v3/users/521f5c381cf44cfeb0e7e2ea6271d404 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:24.388013 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f3b08-5dcf-4510-9035-cbfa70bec38a None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:24.388305 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f3b08-5dcf-4510-9035-cbfa70bec38a None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:24.388736 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f3b08-5dcf-4510-9035-cbfa70bec38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:24.408565 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-727f3b08-5dcf-4510-9035-cbfa70bec38a None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:24.409798 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-727f3b08-5dcf-4510-9035-cbfa70bec38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:24.473369 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1420/2826] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:25:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:24.484830 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b774c1-c893-427f-b09c-0577e4e392e2 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:24.486314 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e3b774c1-c893-427f-b09c-0577e4e392e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:24.548506 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b774c1-c893-427f-b09c-0577e4e392e2 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:24.552283 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64a7b8-3c5e-4c03-8d31-ed8258eaeddd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:24.552930 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64a7b8-3c5e-4c03-8d31-ed8258eaeddd service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:24.553243 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64a7b8-3c5e-4c03-8d31-ed8258eaeddd service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:24.553520 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64a7b8-3c5e-4c03-8d31-ed8258eaeddd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:24.563074 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b64a7b8-3c5e-4c03-8d31-ed8258eaeddd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:24.564515 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b64a7b8-3c5e-4c03-8d31-ed8258eaeddd service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:24.575425 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1407/2827] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:24.885410 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f3b08-5dcf-4510-9035-cbfa70bec38a None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:24.887088 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-727f3b08-5dcf-4510-9035-cbfa70bec38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:24.900897 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f3b08-5dcf-4510-9035-cbfa70bec38a None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:24.907167 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6cbd8b-8c4a-42ea-bf26-66dcf4ee0353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:24.908227 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cbd8b-8c4a-42ea-bf26-66dcf4ee0353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:24.908588 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cbd8b-8c4a-42ea-bf26-66dcf4ee0353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:24.908877 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cbd8b-8c4a-42ea-bf26-66dcf4ee0353 admin admin] PATH_INFO: `/v3/projects/45ef41dbf1d74072a0147fd501819e5a` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:24.910151 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6cbd8b-8c4a-42ea-bf26-66dcf4ee0353 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45ef41dbf1d74072a0147fd501819e5a)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:24.912206 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6cbd8b-8c4a-42ea-bf26-66dcf4ee0353 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:24.913861 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:24.975938 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-4b6cbd8b-8c4a-42ea-bf26-66dcf4ee0353 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45ef41dbf1d74072a0147fd501819e5a', 'request_id': 'req-4b6cbd8b-8c4a-42ea-bf26-66dcf4ee0353'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:24.982786 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-4b6cbd8b-8c4a-42ea-bf26-66dcf4ee0353 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45ef41dbf1d74072a0147fd501819e5a', 'request_id': 'req-4b6cbd8b-8c4a-42ea-bf26-66dcf4ee0353'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:24.990867 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-4b6cbd8b-8c4a-42ea-bf26-66dcf4ee0353 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45ef41dbf1d74072a0147fd501819e5a', 'request_id': 'req-4b6cbd8b-8c4a-42ea-bf26-66dcf4ee0353'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:25.003764 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-4b6cbd8b-8c4a-42ea-bf26-66dcf4ee0353 admin admin] The token cache is being invalidate because project 45ef41dbf1d74072a0147fd501819e5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:25:25.004053 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-4b6cbd8b-8c4a-42ea-bf26-66dcf4ee0353 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:25.004688 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-4b6cbd8b-8c4a-42ea-bf26-66dcf4ee0353 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:25.006262 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1421/2828] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:25:24 2022] DELETE /identity/v3/projects/45ef41dbf1d74072a0147fd501819e5a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:25.010679 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64a7b8-3c5e-4c03-8d31-ed8258eaeddd service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:25.020802 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64a7b8-3c5e-4c03-8d31-ed8258eaeddd service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:25.081396 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1bae74-fa42-48c3-a024-df5769df624e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:25.082315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1bae74-fa42-48c3-a024-df5769df624e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:25.082603 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1bae74-fa42-48c3-a024-df5769df624e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:25.082883 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1bae74-fa42-48c3-a024-df5769df624e admin admin] PATH_INFO: `/v3/users/539992efa88148a49ff7226414cc6994` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:25.084091 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1bae74-fa42-48c3-a024-df5769df624e admin admin] RBAC: Authorizing `identity:delete_user(user_id=539992efa88148a49ff7226414cc6994)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:25.086262 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1bae74-fa42-48c3-a024-df5769df624e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:25.128904 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:25.139887 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7f1bae74-fa42-48c3-a024-df5769df624e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '539992efa88148a49ff7226414cc6994'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:25.152503 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7f1bae74-fa42-48c3-a024-df5769df624e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '539992efa88148a49ff7226414cc6994'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:25.162280 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7f1bae74-fa42-48c3-a024-df5769df624e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '539992efa88148a49ff7226414cc6994'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:25.172375 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7f1bae74-fa42-48c3-a024-df5769df624e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '539992efa88148a49ff7226414cc6994'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:25.173380 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7f1bae74-fa42-48c3-a024-df5769df624e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '539992efa88148a49ff7226414cc6994'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:25.179660 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1408/2829] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:25:25 2022] DELETE /identity/v3/users/539992efa88148a49ff7226414cc6994 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:25.479322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6cbd8b-8c4a-42ea-bf26-66dcf4ee0353 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:25.481075 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-4b6cbd8b-8c4a-42ea-bf26-66dcf4ee0353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:25.502131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6cbd8b-8c4a-42ea-bf26-66dcf4ee0353 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:25.570445 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a336ce-3066-4601-bcd3-955c10302d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:25.571402 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a336ce-3066-4601-bcd3-955c10302d80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:25.571830 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a336ce-3066-4601-bcd3-955c10302d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:25.572311 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a336ce-3066-4601-bcd3-955c10302d80 admin admin] PATH_INFO: `/v3/projects/4b4312f5d3934e3ba708c5e25c77a30e` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:25.573678 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a336ce-3066-4601-bcd3-955c10302d80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b4312f5d3934e3ba708c5e25c77a30e)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:25.575699 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a336ce-3066-4601-bcd3-955c10302d80 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:25.630734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-43a336ce-3066-4601-bcd3-955c10302d80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b4312f5d3934e3ba708c5e25c77a30e', 'request_id': 'req-43a336ce-3066-4601-bcd3-955c10302d80'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:25.637377 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-43a336ce-3066-4601-bcd3-955c10302d80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b4312f5d3934e3ba708c5e25c77a30e', 'request_id': 'req-43a336ce-3066-4601-bcd3-955c10302d80'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:25.647681 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-43a336ce-3066-4601-bcd3-955c10302d80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b4312f5d3934e3ba708c5e25c77a30e', 'request_id': 'req-43a336ce-3066-4601-bcd3-955c10302d80'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:25.659650 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-43a336ce-3066-4601-bcd3-955c10302d80 admin admin] The token cache is being invalidate because project 4b4312f5d3934e3ba708c5e25c77a30e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:25:25.659945 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-43a336ce-3066-4601-bcd3-955c10302d80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:25.660734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-43a336ce-3066-4601-bcd3-955c10302d80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:25.662371 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1422/2830] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:25:25 2022] DELETE /identity/v3/projects/4b4312f5d3934e3ba708c5e25c77a30e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:25.667499 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5671a1-0340-432a-a29c-752dd40cd5e3 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:25.668064 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5671a1-0340-432a-a29c-752dd40cd5e3 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:25.668064 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5671a1-0340-432a-a29c-752dd40cd5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:25.690521 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4b5671a1-0340-432a-a29c-752dd40cd5e3 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:25.691702 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4b5671a1-0340-432a-a29c-752dd40cd5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:25.757770 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1409/2831] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:25:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:25.767136 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01a271-2db9-457d-9f38-89d36ac52ba7 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:25.767330 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01a271-2db9-457d-9f38-89d36ac52ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:25.767585 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01a271-2db9-457d-9f38-89d36ac52ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:25.787687 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-5f01a271-2db9-457d-9f38-89d36ac52ba7 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:25.789246 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-5f01a271-2db9-457d-9f38-89d36ac52ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:25.798068 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1423/2832] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:25:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:25.802577 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5671a1-0340-432a-a29c-752dd40cd5e3 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:25.815223 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5671a1-0340-432a-a29c-752dd40cd5e3 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:25.820009 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda611d0-7093-4b1e-be15-0a06b4a53f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:25.820675 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda611d0-7093-4b1e-be15-0a06b4a53f8b admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:25.820960 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda611d0-7093-4b1e-be15-0a06b4a53f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:25.821236 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda611d0-7093-4b1e-be15-0a06b4a53f8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:25.822095 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda611d0-7093-4b1e-be15-0a06b4a53f8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:25:25.822311 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda611d0-7093-4b1e-be15-0a06b4a53f8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:25.823929 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda611d0-7093-4b1e-be15-0a06b4a53f8b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:25.836640 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1410/2833] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:25:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:25:25.841363 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01a271-2db9-457d-9f38-89d36ac52ba7 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:25.851211 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01a271-2db9-457d-9f38-89d36ac52ba7 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:25.856039 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3316be-3e90-4280-a60e-15618b9b919d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:25.857439 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3316be-3e90-4280-a60e-15618b9b919d admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:25.857979 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3316be-3e90-4280-a60e-15618b9b919d admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:25.857979 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3316be-3e90-4280-a60e-15618b9b919d admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:25.858581 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3316be-3e90-4280-a60e-15618b9b919d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:25.860509 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3316be-3e90-4280-a60e-15618b9b919d admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:25.871246 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:25.890353 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1424/2834] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:25:25 2022] POST /identity/v3/projects => generated 401 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:25.894956 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda611d0-7093-4b1e-be15-0a06b4a53f8b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:25.905324 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda611d0-7093-4b1e-be15-0a06b4a53f8b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:25.969975 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4acc6-564d-409a-8bc7-f3a5ed1c9c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:25.970659 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4acc6-564d-409a-8bc7-f3a5ed1c9c5d admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:25.970946 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4acc6-564d-409a-8bc7-f3a5ed1c9c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:25.971221 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4acc6-564d-409a-8bc7-f3a5ed1c9c5d admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:25.971916 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4acc6-564d-409a-8bc7-f3a5ed1c9c5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:25.973839 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4acc6-564d-409a-8bc7-f3a5ed1c9c5d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:25.995858 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1411/2835] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:25:25 2022] POST /identity/v3/users => generated 301 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:26.000969 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3316be-3e90-4280-a60e-15618b9b919d admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:26.014143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3316be-3e90-4280-a60e-15618b9b919d admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:26.018919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32987599-8df5-4e1f-8118-95d67a5601d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:26.019593 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32987599-8df5-4e1f-8118-95d67a5601d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:26.019877 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32987599-8df5-4e1f-8118-95d67a5601d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:26.020158 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32987599-8df5-4e1f-8118-95d67a5601d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:26.020892 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32987599-8df5-4e1f-8118-95d67a5601d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:26.022537 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32987599-8df5-4e1f-8118-95d67a5601d5 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:26.036925 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1425/2836] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:25:26.041099 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4acc6-564d-409a-8bc7-f3a5ed1c9c5d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:26.052600 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4acc6-564d-409a-8bc7-f3a5ed1c9c5d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:26.057346 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a282d-bbc1-48d5-9159-c42a649dee37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:26.058052 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a282d-bbc1-48d5-9159-c42a649dee37 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:26.058339 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a282d-bbc1-48d5-9159-c42a649dee37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:26.058615 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a282d-bbc1-48d5-9159-c42a649dee37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:26.059352 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278a282d-bbc1-48d5-9159-c42a649dee37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:26.061237 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278a282d-bbc1-48d5-9159-c42a649dee37 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:26.072105 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1412/2837] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:25:26.076789 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32987599-8df5-4e1f-8118-95d67a5601d5 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:26.087433 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32987599-8df5-4e1f-8118-95d67a5601d5 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:26.093343 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6fdef0-8880-4009-a4c0-cc1512a63214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:26.094326 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fdef0-8880-4009-a4c0-cc1512a63214 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:26.094618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fdef0-8880-4009-a4c0-cc1512a63214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:26.094932 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fdef0-8880-4009-a4c0-cc1512a63214 admin admin] PATH_INFO: `/v3/projects/c68641aa1fbb4bbfb66ccfaac1de3c12/users/3cd9a87677f247b3a31d558d325776e8/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:26.112080 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6fdef0-8880-4009-a4c0-cc1512a63214 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c68641aa1fbb4bbfb66ccfaac1de3c12, user_id=3cd9a87677f247b3a31d558d325776e8, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:26.114918 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6fdef0-8880-4009-a4c0-cc1512a63214 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:26.135470 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1426/2838] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:25:26 2022] PUT /identity/v3/projects/c68641aa1fbb4bbfb66ccfaac1de3c12/users/3cd9a87677f247b3a31d558d325776e8/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:26.154557 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d03d4-f7c2-4711-a8a8-2de28e5fb42e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:26.154845 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d03d4-f7c2-4711-a8a8-2de28e5fb42e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:26.155183 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d03d4-f7c2-4711-a8a8-2de28e5fb42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:26.186118 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1d5d03d4-f7c2-4711-a8a8-2de28e5fb42e None None] MFA Rules not processed for user `3cd9a87677f247b3a31d558d325776e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:26.187341 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1d5d03d4-f7c2-4711-a8a8-2de28e5fb42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:26.196179 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:26.258844 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1413/2839] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:25:26 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:26.263613 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6fdef0-8880-4009-a4c0-cc1512a63214 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:26.329295 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6fdef0-8880-4009-a4c0-cc1512a63214 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:26.334395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fcafa3-c670-4412-aa63-9ba429e8e8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:26.335079 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcafa3-c670-4412-aa63-9ba429e8e8ce admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:26.335391 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcafa3-c670-4412-aa63-9ba429e8e8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:26.335627 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcafa3-c670-4412-aa63-9ba429e8e8ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:26.336334 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fcafa3-c670-4412-aa63-9ba429e8e8ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:26.338349 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fcafa3-c670-4412-aa63-9ba429e8e8ce admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:26.365053 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1427/2840] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:25:26 2022] POST /identity/v3/projects => generated 401 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:26.369204 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d03d4-f7c2-4711-a8a8-2de28e5fb42e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:26.380706 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d03d4-f7c2-4711-a8a8-2de28e5fb42e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:26.453162 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab18e3-4396-4554-9eab-cf1e78946ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:26.453742 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab18e3-4396-4554-9eab-cf1e78946ea7 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:26.454077 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab18e3-4396-4554-9eab-cf1e78946ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:26.454352 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab18e3-4396-4554-9eab-cf1e78946ea7 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:26.455061 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab18e3-4396-4554-9eab-cf1e78946ea7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:26.457032 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab18e3-4396-4554-9eab-cf1e78946ea7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:26.479868 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1414/2841] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:25:26 2022] POST /identity/v3/users => generated 307 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:26.484111 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fcafa3-c670-4412-aa63-9ba429e8e8ce admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:26.495322 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fcafa3-c670-4412-aa63-9ba429e8e8ce admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:26.500093 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c753d-a347-4770-a725-5206784b135c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:26.500778 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c753d-a347-4770-a725-5206784b135c admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:26.501059 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c753d-a347-4770-a725-5206784b135c admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:26.501400 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c753d-a347-4770-a725-5206784b135c admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:26.502173 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c753d-a347-4770-a725-5206784b135c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:26.503804 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c753d-a347-4770-a725-5206784b135c admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:26.515957 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1428/2842] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:25:26.520243 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab18e3-4396-4554-9eab-cf1e78946ea7 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:26.531731 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab18e3-4396-4554-9eab-cf1e78946ea7 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:26.536845 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e16b5-0bc2-4461-a651-d8bd55b43df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:26.537772 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e16b5-0bc2-4461-a651-d8bd55b43df2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:26.538195 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e16b5-0bc2-4461-a651-d8bd55b43df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:26.538477 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e16b5-0bc2-4461-a651-d8bd55b43df2 admin admin] PATH_INFO: `/v3/projects/5383c7ead40a4084b349e46066ed9999/users/7181d5cd9bf14ff9b6a11f908b12b252/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:26.554602 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e16b5-0bc2-4461-a651-d8bd55b43df2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5383c7ead40a4084b349e46066ed9999, user_id=7181d5cd9bf14ff9b6a11f908b12b252, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:26.557353 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e16b5-0bc2-4461-a651-d8bd55b43df2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:26.572343 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1415/2843] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:25:26 2022] PUT /identity/v3/projects/5383c7ead40a4084b349e46066ed9999/users/7181d5cd9bf14ff9b6a11f908b12b252/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:26.577074 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c753d-a347-4770-a725-5206784b135c admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:26.589086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c753d-a347-4770-a725-5206784b135c admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:26.647935 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c9d2f-4737-4260-ad4d-f012b3578cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:26.648606 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c9d2f-4737-4260-ad4d-f012b3578cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:26.648887 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c9d2f-4737-4260-ad4d-f012b3578cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:26.649185 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c9d2f-4737-4260-ad4d-f012b3578cd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:26.649883 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c9d2f-4737-4260-ad4d-f012b3578cd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:26.651519 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c9d2f-4737-4260-ad4d-f012b3578cd0 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:26.663394 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1429/2844] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:25:26.667641 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e16b5-0bc2-4461-a651-d8bd55b43df2 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:26.679805 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e16b5-0bc2-4461-a651-d8bd55b43df2 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:26.685324 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586d1b3-47c7-4cf3-b719-5b3ca461ce76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:26.686466 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586d1b3-47c7-4cf3-b719-5b3ca461ce76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:26.686751 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586d1b3-47c7-4cf3-b719-5b3ca461ce76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:26.687061 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586d1b3-47c7-4cf3-b719-5b3ca461ce76 admin admin] PATH_INFO: `/v3/projects/5383c7ead40a4084b349e46066ed9999/users/7181d5cd9bf14ff9b6a11f908b12b252/roles/bbbc77ec645d4ff5a8bf29ce0377f578` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:26.689133 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e586d1b3-47c7-4cf3-b719-5b3ca461ce76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5383c7ead40a4084b349e46066ed9999, user_id=7181d5cd9bf14ff9b6a11f908b12b252, role_id=bbbc77ec645d4ff5a8bf29ce0377f578)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:26.691975 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e586d1b3-47c7-4cf3-b719-5b3ca461ce76 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:26.704298 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1416/2845] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:25:26 2022] PUT /identity/v3/projects/5383c7ead40a4084b349e46066ed9999/users/7181d5cd9bf14ff9b6a11f908b12b252/roles/bbbc77ec645d4ff5a8bf29ce0377f578 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:25:26.715400 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2573a-86fd-4c10-9719-3d9b57113319 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:26.715694 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2573a-86fd-4c10-9719-3d9b57113319 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:26.716115 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2573a-86fd-4c10-9719-3d9b57113319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:26.746329 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-2af2573a-86fd-4c10-9719-3d9b57113319 None None] MFA Rules not processed for user `7181d5cd9bf14ff9b6a11f908b12b252`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:26.748596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-2af2573a-86fd-4c10-9719-3d9b57113319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:26.761587 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:26.822648 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1430/2846] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:25:26 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:26.834438 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586d1b3-47c7-4cf3-b719-5b3ca461ce76 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:26.836162 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e586d1b3-47c7-4cf3-b719-5b3ca461ce76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:26.898696 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586d1b3-47c7-4cf3-b719-5b3ca461ce76 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:26.902406 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb67fa-6290-4c7a-978c-c8174df88517 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:26.903062 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb67fa-6290-4c7a-978c-c8174df88517 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:26.903501 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb67fa-6290-4c7a-978c-c8174df88517 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:26.903975 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb67fa-6290-4c7a-978c-c8174df88517 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:26.913607 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feb67fa-6290-4c7a-978c-c8174df88517 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:26.915104 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feb67fa-6290-4c7a-978c-c8174df88517 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:26.925265 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1417/2847] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:26 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:27.000128 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2573a-86fd-4c10-9719-3d9b57113319 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:27.010401 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2573a-86fd-4c10-9719-3d9b57113319 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:27.014857 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe5ade-3fd6-40bd-a44f-d817bcfa6e5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:27.015501 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe5ade-3fd6-40bd-a44f-d817bcfa6e5d service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:27.015782 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe5ade-3fd6-40bd-a44f-d817bcfa6e5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:27.016061 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe5ade-3fd6-40bd-a44f-d817bcfa6e5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:27.075648 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebe5ade-3fd6-40bd-a44f-d817bcfa6e5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:27.077109 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebe5ade-3fd6-40bd-a44f-d817bcfa6e5d service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:27.086121 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1431/2848] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:26 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:31.240151 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb67fa-6290-4c7a-978c-c8174df88517 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:31.245892 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:31.324797 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb67fa-6290-4c7a-978c-c8174df88517 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:31.330803 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09486a54-b323-42bc-a941-f65373683ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:31.331695 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09486a54-b323-42bc-a941-f65373683ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:31.332003 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09486a54-b323-42bc-a941-f65373683ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:31.332277 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09486a54-b323-42bc-a941-f65373683ec8 admin admin] PATH_INFO: `/v3/users/3cd9a87677f247b3a31d558d325776e8` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:31.333465 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09486a54-b323-42bc-a941-f65373683ec8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cd9a87677f247b3a31d558d325776e8)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:31.335456 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09486a54-b323-42bc-a941-f65373683ec8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:31.386209 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-09486a54-b323-42bc-a941-f65373683ec8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cd9a87677f247b3a31d558d325776e8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:31.398863 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-09486a54-b323-42bc-a941-f65373683ec8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cd9a87677f247b3a31d558d325776e8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:31.406607 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-09486a54-b323-42bc-a941-f65373683ec8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cd9a87677f247b3a31d558d325776e8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:31.413813 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-09486a54-b323-42bc-a941-f65373683ec8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cd9a87677f247b3a31d558d325776e8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:31.414472 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-09486a54-b323-42bc-a941-f65373683ec8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cd9a87677f247b3a31d558d325776e8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:31.420371 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1418/2849] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:25:31 2022] DELETE /identity/v3/users/3cd9a87677f247b3a31d558d325776e8 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:31.426081 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e0a34-d4e5-4b74-bf7c-78eedf43a0cf None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:31.426081 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e0a34-d4e5-4b74-bf7c-78eedf43a0cf None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:31.426081 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e0a34-d4e5-4b74-bf7c-78eedf43a0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:31.444362 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-199e0a34-d4e5-4b74-bf7c-78eedf43a0cf None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:31.445600 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-199e0a34-d4e5-4b74-bf7c-78eedf43a0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:31.510330 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1432/2850] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:25:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:25:31.521394 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09486a54-b323-42bc-a941-f65373683ec8 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:31.522892 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-09486a54-b323-42bc-a941-f65373683ec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:31.583451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09486a54-b323-42bc-a941-f65373683ec8 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:31.587182 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e89b5-63b7-481a-b966-6bb8b0ccf5a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:31.587840 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e89b5-63b7-481a-b966-6bb8b0ccf5a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:31.588127 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e89b5-63b7-481a-b966-6bb8b0ccf5a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:31.588407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e89b5-63b7-481a-b966-6bb8b0ccf5a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:31.597649 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e89b5-63b7-481a-b966-6bb8b0ccf5a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:31.599120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e89b5-63b7-481a-b966-6bb8b0ccf5a8 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:31.609451 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1419/2851] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:31.898987 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e0a34-d4e5-4b74-bf7c-78eedf43a0cf None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:31.901509 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-199e0a34-d4e5-4b74-bf7c-78eedf43a0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:31.920086 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e0a34-d4e5-4b74-bf7c-78eedf43a0cf None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:31.926334 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc0297-594f-4b28-8166-8880d9ebf7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:31.927251 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc0297-594f-4b28-8166-8880d9ebf7a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:31.927537 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc0297-594f-4b28-8166-8880d9ebf7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:31.927926 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc0297-594f-4b28-8166-8880d9ebf7a7 admin admin] PATH_INFO: `/v3/projects/c68641aa1fbb4bbfb66ccfaac1de3c12` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:31.929633 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dc0297-594f-4b28-8166-8880d9ebf7a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c68641aa1fbb4bbfb66ccfaac1de3c12)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:31.931908 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dc0297-594f-4b28-8166-8880d9ebf7a7 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:31.982641 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:31.986885 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-04dc0297-594f-4b28-8166-8880d9ebf7a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c68641aa1fbb4bbfb66ccfaac1de3c12', 'request_id': 'req-04dc0297-594f-4b28-8166-8880d9ebf7a7'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:31.993785 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-04dc0297-594f-4b28-8166-8880d9ebf7a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c68641aa1fbb4bbfb66ccfaac1de3c12', 'request_id': 'req-04dc0297-594f-4b28-8166-8880d9ebf7a7'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:32.001361 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-04dc0297-594f-4b28-8166-8880d9ebf7a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c68641aa1fbb4bbfb66ccfaac1de3c12', 'request_id': 'req-04dc0297-594f-4b28-8166-8880d9ebf7a7'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:32.013890 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-04dc0297-594f-4b28-8166-8880d9ebf7a7 admin admin] The token cache is being invalidate because project c68641aa1fbb4bbfb66ccfaac1de3c12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:25:32.014234 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-04dc0297-594f-4b28-8166-8880d9ebf7a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:32.015504 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-04dc0297-594f-4b28-8166-8880d9ebf7a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:32.017072 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1433/2852] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:25:31 2022] DELETE /identity/v3/projects/c68641aa1fbb4bbfb66ccfaac1de3c12 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:32.021063 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e89b5-63b7-481a-b966-6bb8b0ccf5a8 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:32.031715 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e89b5-63b7-481a-b966-6bb8b0ccf5a8 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:32.056785 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:32.100246 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f2baf-29ed-4252-b5e1-9090bf5b3738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:32.101045 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f2baf-29ed-4252-b5e1-9090bf5b3738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:32.101335 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f2baf-29ed-4252-b5e1-9090bf5b3738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:32.101711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f2baf-29ed-4252-b5e1-9090bf5b3738 admin admin] PATH_INFO: `/v3/users/7181d5cd9bf14ff9b6a11f908b12b252` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:32.103005 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f2baf-29ed-4252-b5e1-9090bf5b3738 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7181d5cd9bf14ff9b6a11f908b12b252)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:32.105047 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f2baf-29ed-4252-b5e1-9090bf5b3738 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:32.157632 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d87f2baf-29ed-4252-b5e1-9090bf5b3738 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7181d5cd9bf14ff9b6a11f908b12b252'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:32.173490 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d87f2baf-29ed-4252-b5e1-9090bf5b3738 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7181d5cd9bf14ff9b6a11f908b12b252'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:32.182558 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d87f2baf-29ed-4252-b5e1-9090bf5b3738 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7181d5cd9bf14ff9b6a11f908b12b252'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:32.190364 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d87f2baf-29ed-4252-b5e1-9090bf5b3738 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7181d5cd9bf14ff9b6a11f908b12b252'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:32.191056 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d87f2baf-29ed-4252-b5e1-9090bf5b3738 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7181d5cd9bf14ff9b6a11f908b12b252'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:32.199324 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1420/2853] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:25:32 2022] DELETE /identity/v3/users/7181d5cd9bf14ff9b6a11f908b12b252 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:32.499530 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc0297-594f-4b28-8166-8880d9ebf7a7 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:32.501009 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-04dc0297-594f-4b28-8166-8880d9ebf7a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:32.518742 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc0297-594f-4b28-8166-8880d9ebf7a7 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:32.584460 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f4886-24b3-41f6-876e-36dd09b07a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:32.585335 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f4886-24b3-41f6-876e-36dd09b07a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:32.585623 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f4886-24b3-41f6-876e-36dd09b07a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:32.585906 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f4886-24b3-41f6-876e-36dd09b07a3f admin admin] PATH_INFO: `/v3/projects/5383c7ead40a4084b349e46066ed9999` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:32.587208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f4886-24b3-41f6-876e-36dd09b07a3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5383c7ead40a4084b349e46066ed9999)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:32.589166 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f4886-24b3-41f6-876e-36dd09b07a3f admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:32.640206 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-ef4f4886-24b3-41f6-876e-36dd09b07a3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5383c7ead40a4084b349e46066ed9999', 'request_id': 'req-ef4f4886-24b3-41f6-876e-36dd09b07a3f'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:32.646801 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-ef4f4886-24b3-41f6-876e-36dd09b07a3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5383c7ead40a4084b349e46066ed9999', 'request_id': 'req-ef4f4886-24b3-41f6-876e-36dd09b07a3f'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:32.657018 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-ef4f4886-24b3-41f6-876e-36dd09b07a3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5383c7ead40a4084b349e46066ed9999', 'request_id': 'req-ef4f4886-24b3-41f6-876e-36dd09b07a3f'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:32.668822 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-ef4f4886-24b3-41f6-876e-36dd09b07a3f admin admin] The token cache is being invalidate because project 5383c7ead40a4084b349e46066ed9999 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:25:32.669113 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-ef4f4886-24b3-41f6-876e-36dd09b07a3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:32.669919 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-ef4f4886-24b3-41f6-876e-36dd09b07a3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:32.671641 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1434/2854] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:25:32 2022] DELETE /identity/v3/projects/5383c7ead40a4084b349e46066ed9999 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:32.676714 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02b413-d7f3-4228-84af-80f453aefe25 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:32.677559 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02b413-d7f3-4228-84af-80f453aefe25 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:32.677559 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02b413-d7f3-4228-84af-80f453aefe25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:32.697447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-da02b413-d7f3-4228-84af-80f453aefe25 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:32.698717 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-da02b413-d7f3-4228-84af-80f453aefe25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:32.773752 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1421/2855] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:25:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:32.782985 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018d3d9-aeb9-4d89-8cae-5ca2ed655212 None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:32.783281 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018d3d9-aeb9-4d89-8cae-5ca2ed655212 None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:32.783563 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018d3d9-aeb9-4d89-8cae-5ca2ed655212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:32.802235 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-3018d3d9-aeb9-4d89-8cae-5ca2ed655212 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:32.803448 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-3018d3d9-aeb9-4d89-8cae-5ca2ed655212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:32.812004 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1435/2856] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:25:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:32.816395 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02b413-d7f3-4228-84af-80f453aefe25 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:32.826129 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02b413-d7f3-4228-84af-80f453aefe25 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:32.831695 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7594b-d64a-42e6-a064-7cf9e350d89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:32.832407 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7594b-d64a-42e6-a064-7cf9e350d89e admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:32.832694 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7594b-d64a-42e6-a064-7cf9e350d89e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:32.832976 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7594b-d64a-42e6-a064-7cf9e350d89e admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:32.833711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf7594b-d64a-42e6-a064-7cf9e350d89e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:25:32.834055 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf7594b-d64a-42e6-a064-7cf9e350d89e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:32.835663 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf7594b-d64a-42e6-a064-7cf9e350d89e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:32.848113 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1422/2857] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:25:32.904084 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018d3d9-aeb9-4d89-8cae-5ca2ed655212 None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:32.914512 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018d3d9-aeb9-4d89-8cae-5ca2ed655212 None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:32.919527 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df97bbd6-8e85-4346-b3ef-79058098f455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:32.920241 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97bbd6-8e85-4346-b3ef-79058098f455 admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:32.920570 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97bbd6-8e85-4346-b3ef-79058098f455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:32.920851 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97bbd6-8e85-4346-b3ef-79058098f455 admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:32.921556 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df97bbd6-8e85-4346-b3ef-79058098f455 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:32.923540 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df97bbd6-8e85-4346-b3ef-79058098f455 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:32.950862 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1436/2858] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:25:32 2022] POST /identity/v3/projects => generated 361 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:32.955685 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7594b-d64a-42e6-a064-7cf9e350d89e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:32.966069 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7594b-d64a-42e6-a064-7cf9e350d89e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:33.008610 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:33.042127 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af3464-a6cc-4296-b3b4-7c223b2c7455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:33.042819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af3464-a6cc-4296-b3b4-7c223b2c7455 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:33.043100 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af3464-a6cc-4296-b3b4-7c223b2c7455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:33.043375 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af3464-a6cc-4296-b3b4-7c223b2c7455 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:33.044110 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11af3464-a6cc-4296-b3b4-7c223b2c7455 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:33.046647 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11af3464-a6cc-4296-b3b4-7c223b2c7455 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:33.067829 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1423/2859] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:25:32 2022] POST /identity/v3/users => generated 281 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:25:33.072258 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df97bbd6-8e85-4346-b3ef-79058098f455 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:33.084770 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df97bbd6-8e85-4346-b3ef-79058098f455 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:33.089735 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f60e868-9bfa-4e53-85dd-e0aabe1e1769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:33.090478 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60e868-9bfa-4e53-85dd-e0aabe1e1769 admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:33.090893 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60e868-9bfa-4e53-85dd-e0aabe1e1769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:33.091059 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60e868-9bfa-4e53-85dd-e0aabe1e1769 admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:33.091802 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f60e868-9bfa-4e53-85dd-e0aabe1e1769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:33.093447 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f60e868-9bfa-4e53-85dd-e0aabe1e1769 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:33.105560 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1437/2860] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:25:33.110819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af3464-a6cc-4296-b3b4-7c223b2c7455 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:33.121283 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af3464-a6cc-4296-b3b4-7c223b2c7455 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:33.127816 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71812905-3d87-458f-b93e-8d7fe312352c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:33.128497 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71812905-3d87-458f-b93e-8d7fe312352c admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:33.128783 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71812905-3d87-458f-b93e-8d7fe312352c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:33.129057 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71812905-3d87-458f-b93e-8d7fe312352c admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:33.129787 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71812905-3d87-458f-b93e-8d7fe312352c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:33.131429 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71812905-3d87-458f-b93e-8d7fe312352c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:33.142910 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1424/2861] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:25:33.147240 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f60e868-9bfa-4e53-85dd-e0aabe1e1769 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:33.158263 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f60e868-9bfa-4e53-85dd-e0aabe1e1769 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:33.162993 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa29303-8d91-41c0-bae1-11a3bc523853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:33.163884 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa29303-8d91-41c0-bae1-11a3bc523853 admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:33.164168 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa29303-8d91-41c0-bae1-11a3bc523853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:33.164446 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa29303-8d91-41c0-bae1-11a3bc523853 admin admin] PATH_INFO: `/v3/projects/5606dd7930a443ceacc2af0fe07debe6/users/2c2687987959449291137e045bd0cba4/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:33.181976 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa29303-8d91-41c0-bae1-11a3bc523853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5606dd7930a443ceacc2af0fe07debe6, user_id=2c2687987959449291137e045bd0cba4, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:33.185003 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa29303-8d91-41c0-bae1-11a3bc523853 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:33.196143 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:33.202985 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1438/2862] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:25:33 2022] PUT /identity/v3/projects/5606dd7930a443ceacc2af0fe07debe6/users/2c2687987959449291137e045bd0cba4/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:33.214758 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82576e94-4f4f-448b-8807-186fdb41ede2 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:33.215054 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82576e94-4f4f-448b-8807-186fdb41ede2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:33.215338 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82576e94-4f4f-448b-8807-186fdb41ede2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:33.245725 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-82576e94-4f4f-448b-8807-186fdb41ede2 None None] MFA Rules not processed for user `2c2687987959449291137e045bd0cba4`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:33.247010 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-82576e94-4f4f-448b-8807-186fdb41ede2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:33.324379 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1425/2863] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:25:33 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:33.360275 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa29303-8d91-41c0-bae1-11a3bc523853 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:33.362314 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-0aa29303-8d91-41c0-bae1-11a3bc523853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:33.426537 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa29303-8d91-41c0-bae1-11a3bc523853 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:33.430450 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af20630-28e5-4415-ad6e-05ffb1882f47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:33.431114 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af20630-28e5-4415-ad6e-05ffb1882f47 service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:33.431403 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af20630-28e5-4415-ad6e-05ffb1882f47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:33.431684 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af20630-28e5-4415-ad6e-05ffb1882f47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:33.441335 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af20630-28e5-4415-ad6e-05ffb1882f47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:33.442856 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af20630-28e5-4415-ad6e-05ffb1882f47 service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:33.452953 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1439/2864] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:33 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:33.468193 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82576e94-4f4f-448b-8807-186fdb41ede2 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:33.537399 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82576e94-4f4f-448b-8807-186fdb41ede2 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:33.542383 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e95b40-10cf-4b89-baeb-e1e7b2efe36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:33.543280 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e95b40-10cf-4b89-baeb-e1e7b2efe36b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:33.543565 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e95b40-10cf-4b89-baeb-e1e7b2efe36b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:33.543849 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e95b40-10cf-4b89-baeb-e1e7b2efe36b admin admin] PATH_INFO: `/v3/users/2c2687987959449291137e045bd0cba4` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:33.545134 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e95b40-10cf-4b89-baeb-e1e7b2efe36b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c2687987959449291137e045bd0cba4)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:33.547873 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e95b40-10cf-4b89-baeb-e1e7b2efe36b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:33.597190 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:33.600668 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-86e95b40-10cf-4b89-baeb-e1e7b2efe36b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c2687987959449291137e045bd0cba4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:33.615231 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-86e95b40-10cf-4b89-baeb-e1e7b2efe36b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c2687987959449291137e045bd0cba4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:33.623069 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-86e95b40-10cf-4b89-baeb-e1e7b2efe36b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c2687987959449291137e045bd0cba4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:33.630605 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-86e95b40-10cf-4b89-baeb-e1e7b2efe36b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c2687987959449291137e045bd0cba4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:33.631289 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-86e95b40-10cf-4b89-baeb-e1e7b2efe36b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c2687987959449291137e045bd0cba4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:33.637986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1426/2865] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:25:33 2022] DELETE /identity/v3/users/2c2687987959449291137e045bd0cba4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:33.642739 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be03802-6aab-4eec-928d-0fc5352e76ba None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:33.643573 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be03802-6aab-4eec-928d-0fc5352e76ba None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:33.643573 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be03802-6aab-4eec-928d-0fc5352e76ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:33.669203 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-9be03802-6aab-4eec-928d-0fc5352e76ba None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:33.671192 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-9be03802-6aab-4eec-928d-0fc5352e76ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:33.744766 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1440/2866] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:25:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:25:33.754737 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e95b40-10cf-4b89-baeb-e1e7b2efe36b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:33.756265 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-86e95b40-10cf-4b89-baeb-e1e7b2efe36b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:33.819236 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e95b40-10cf-4b89-baeb-e1e7b2efe36b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:33.823089 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff45c5-cbb4-4e43-a351-c572e65d4802 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:33.823739 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff45c5-cbb4-4e43-a351-c572e65d4802 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:33.824024 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff45c5-cbb4-4e43-a351-c572e65d4802 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:33.824301 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff45c5-cbb4-4e43-a351-c572e65d4802 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:33.834931 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff45c5-cbb4-4e43-a351-c572e65d4802 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:33.836374 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff45c5-cbb4-4e43-a351-c572e65d4802 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:33.846916 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1427/2867] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:25:34.142223 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be03802-6aab-4eec-928d-0fc5352e76ba None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:34.143869 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-9be03802-6aab-4eec-928d-0fc5352e76ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:34.160976 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be03802-6aab-4eec-928d-0fc5352e76ba None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:34.165808 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b1d7c-f0be-4c0b-85a0-e354020d1192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:34.166755 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b1d7c-f0be-4c0b-85a0-e354020d1192 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:34.167010 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b1d7c-f0be-4c0b-85a0-e354020d1192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:34.167284 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b1d7c-f0be-4c0b-85a0-e354020d1192 admin admin] PATH_INFO: `/v3/projects/5606dd7930a443ceacc2af0fe07debe6` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:34.168505 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2b1d7c-f0be-4c0b-85a0-e354020d1192 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5606dd7930a443ceacc2af0fe07debe6)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:34.171449 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2b1d7c-f0be-4c0b-85a0-e354020d1192 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:34.221206 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-4d2b1d7c-f0be-4c0b-85a0-e354020d1192 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5606dd7930a443ceacc2af0fe07debe6', 'request_id': 'req-4d2b1d7c-f0be-4c0b-85a0-e354020d1192'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:34.226926 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-4d2b1d7c-f0be-4c0b-85a0-e354020d1192 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5606dd7930a443ceacc2af0fe07debe6', 'request_id': 'req-4d2b1d7c-f0be-4c0b-85a0-e354020d1192'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:34.235675 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-4d2b1d7c-f0be-4c0b-85a0-e354020d1192 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5606dd7930a443ceacc2af0fe07debe6', 'request_id': 'req-4d2b1d7c-f0be-4c0b-85a0-e354020d1192'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:34.248097 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-4d2b1d7c-f0be-4c0b-85a0-e354020d1192 admin admin] The token cache is being invalidate because project 5606dd7930a443ceacc2af0fe07debe6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:25:34.249146 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-4d2b1d7c-f0be-4c0b-85a0-e354020d1192 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:34.250441 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-4d2b1d7c-f0be-4c0b-85a0-e354020d1192 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:34.252064 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1441/2868] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:25:34 2022] DELETE /identity/v3/projects/5606dd7930a443ceacc2af0fe07debe6 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:34.256996 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b5a62-e76b-446f-9a4b-db7846a4e862 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:34.257292 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b5a62-e76b-446f-9a4b-db7846a4e862 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:34.257580 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b5a62-e76b-446f-9a4b-db7846a4e862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:34.277614 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e44b5a62-e76b-446f-9a4b-db7846a4e862 None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:34.278955 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e44b5a62-e76b-446f-9a4b-db7846a4e862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:34.340202 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1428/2869] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:34.347945 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58534dd4-026b-482b-86c0-8c021822368b None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:34.348230 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58534dd4-026b-482b-86c0-8c021822368b None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:34.348508 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58534dd4-026b-482b-86c0-8c021822368b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:34.367388 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-58534dd4-026b-482b-86c0-8c021822368b None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:34.368618 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-58534dd4-026b-482b-86c0-8c021822368b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:34.377460 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1442/2870] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:34.381952 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b5a62-e76b-446f-9a4b-db7846a4e862 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:34.393358 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b5a62-e76b-446f-9a4b-db7846a4e862 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:34.397866 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4f0c0-a418-46d2-8c41-b720534734d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:34.398551 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4f0c0-a418-46d2-8c41-b720534734d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:34.398859 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4f0c0-a418-46d2-8c41-b720534734d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:34.399135 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4f0c0-a418-46d2-8c41-b720534734d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:34.399878 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4f0c0-a418-46d2-8c41-b720534734d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:25:34.400180 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4f0c0-a418-46d2-8c41-b720534734d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:34.401809 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4f0c0-a418-46d2-8c41-b720534734d2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:34.414666 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1429/2871] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 16:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:25:34.419174 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58534dd4-026b-482b-86c0-8c021822368b None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:34.429753 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58534dd4-026b-482b-86c0-8c021822368b None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:34.434513 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa4270-51fc-4994-baaa-46cab7c04a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:34.435175 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa4270-51fc-4994-baaa-46cab7c04a0e admin admin] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:34.435462 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa4270-51fc-4994-baaa-46cab7c04a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:34.435742 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa4270-51fc-4994-baaa-46cab7c04a0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:34.436732 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa4270-51fc-4994-baaa-46cab7c04a0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:34.438489 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa4270-51fc-4994-baaa-46cab7c04a0e admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:34.458091 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:34.466296 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1443/2872] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 16:25:34 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:34.471152 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4f0c0-a418-46d2-8c41-b720534734d2 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:34.481496 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4f0c0-a418-46d2-8c41-b720534734d2 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:34.539283 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:34.551094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f1068-3e20-4fa1-be9c-98b7c96b1b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:34.551946 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f1068-3e20-4fa1-be9c-98b7c96b1b07 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:34.552234 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f1068-3e20-4fa1-be9c-98b7c96b1b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:34.552516 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f1068-3e20-4fa1-be9c-98b7c96b1b07 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:34.553223 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f1068-3e20-4fa1-be9c-98b7c96b1b07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:34.555179 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f1068-3e20-4fa1-be9c-98b7c96b1b07 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:34.575218 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1430/2873] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 16:25:34 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:34.581189 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa4270-51fc-4994-baaa-46cab7c04a0e admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:34.593471 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa4270-51fc-4994-baaa-46cab7c04a0e admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:34.598522 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00e615-86f1-4b06-b193-b4f4237002fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:34.599213 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00e615-86f1-4b06-b193-b4f4237002fb admin admin] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:34.599502 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00e615-86f1-4b06-b193-b4f4237002fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:34.599783 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00e615-86f1-4b06-b193-b4f4237002fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:34.600522 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00e615-86f1-4b06-b193-b4f4237002fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:34.602208 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00e615-86f1-4b06-b193-b4f4237002fb admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:34.613785 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1444/2874] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:25:34.618602 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f1068-3e20-4fa1-be9c-98b7c96b1b07 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:34.630115 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f1068-3e20-4fa1-be9c-98b7c96b1b07 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:34.635320 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d1239-dc0d-46a6-a386-eea22a635f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:34.635995 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d1239-dc0d-46a6-a386-eea22a635f54 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:34.636540 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d1239-dc0d-46a6-a386-eea22a635f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:34.636540 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d1239-dc0d-46a6-a386-eea22a635f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:34.637258 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d1239-dc0d-46a6-a386-eea22a635f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:34.638899 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d1239-dc0d-46a6-a386-eea22a635f54 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:34.652316 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1431/2875] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:25:34.657094 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00e615-86f1-4b06-b193-b4f4237002fb admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:34.672060 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00e615-86f1-4b06-b193-b4f4237002fb admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:34.677080 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cedf0af-28fe-4e79-abe6-beba396a2d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:34.678031 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cedf0af-28fe-4e79-abe6-beba396a2d9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:34.678317 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cedf0af-28fe-4e79-abe6-beba396a2d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:34.678595 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cedf0af-28fe-4e79-abe6-beba396a2d9e admin admin] PATH_INFO: `/v3/projects/71b592d6be0b4f9a9bf9c884bc21097b/users/da5dfc8a3848443e9d4593be15601ead/roles/3ce570490865493380e6941d004136bd` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:34.696596 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cedf0af-28fe-4e79-abe6-beba396a2d9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b592d6be0b4f9a9bf9c884bc21097b, user_id=da5dfc8a3848443e9d4593be15601ead, role_id=3ce570490865493380e6941d004136bd)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:34.699358 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cedf0af-28fe-4e79-abe6-beba396a2d9e admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:34.715059 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1445/2876] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 16:25:34 2022] PUT /identity/v3/projects/71b592d6be0b4f9a9bf9c884bc21097b/users/da5dfc8a3848443e9d4593be15601ead/roles/3ce570490865493380e6941d004136bd => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:34.729201 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cec8e-239f-4412-9e58-aa6fe3974b11 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:34.729493 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cec8e-239f-4412-9e58-aa6fe3974b11 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:34.729777 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cec8e-239f-4412-9e58-aa6fe3974b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:34.760742 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d07cec8e-239f-4412-9e58-aa6fe3974b11 None None] MFA Rules not processed for user `da5dfc8a3848443e9d4593be15601ead`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:34.762013 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d07cec8e-239f-4412-9e58-aa6fe3974b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:34.834507 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1432/2877] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:25:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:34.846445 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cedf0af-28fe-4e79-abe6-beba396a2d9e admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:34.848928 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-6cedf0af-28fe-4e79-abe6-beba396a2d9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:34.915029 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cedf0af-28fe-4e79-abe6-beba396a2d9e admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:34.919500 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edd8e52-cf16-4324-a8b3-70a73c69bce5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:34.920491 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd8e52-cf16-4324-a8b3-70a73c69bce5 service cinder] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:34.920924 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd8e52-cf16-4324-a8b3-70a73c69bce5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:34.921407 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd8e52-cf16-4324-a8b3-70a73c69bce5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:34.930933 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edd8e52-cf16-4324-a8b3-70a73c69bce5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:34.932488 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edd8e52-cf16-4324-a8b3-70a73c69bce5 service cinder] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:34.943557 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1446/2878] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:43.969399 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07cec8e-239f-4412-9e58-aa6fe3974b11 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:44.036032 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07cec8e-239f-4412-9e58-aa6fe3974b11 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:44.041265 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5210305-f089-4025-b1e9-de371741a534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:44.042134 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5210305-f089-4025-b1e9-de371741a534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:44.042425 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5210305-f089-4025-b1e9-de371741a534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:44.042704 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5210305-f089-4025-b1e9-de371741a534 admin admin] PATH_INFO: `/v3/users/da5dfc8a3848443e9d4593be15601ead` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:44.043895 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5210305-f089-4025-b1e9-de371741a534 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da5dfc8a3848443e9d4593be15601ead)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:44.045832 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5210305-f089-4025-b1e9-de371741a534 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:44.095315 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c5210305-f089-4025-b1e9-de371741a534 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da5dfc8a3848443e9d4593be15601ead'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:44.105583 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:44.108237 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c5210305-f089-4025-b1e9-de371741a534 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da5dfc8a3848443e9d4593be15601ead'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:44.119204 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c5210305-f089-4025-b1e9-de371741a534 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da5dfc8a3848443e9d4593be15601ead'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:44.128819 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c5210305-f089-4025-b1e9-de371741a534 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da5dfc8a3848443e9d4593be15601ead'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:44.129550 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c5210305-f089-4025-b1e9-de371741a534 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da5dfc8a3848443e9d4593be15601ead'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:44.135757 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1433/2879] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:25:43 2022] DELETE /identity/v3/users/da5dfc8a3848443e9d4593be15601ead => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:44.140433 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca95592-48f7-43f4-b0fa-394ffbb9b61b None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:44.140989 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca95592-48f7-43f4-b0fa-394ffbb9b61b None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:44.141112 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca95592-48f7-43f4-b0fa-394ffbb9b61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:44.161054 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-5ca95592-48f7-43f4-b0fa-394ffbb9b61b None None] MFA Rules not processed for user `9593d0f94eed4e3e83f1323aab2716a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:44.162305 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-5ca95592-48f7-43f4-b0fa-394ffbb9b61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:44.224743 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1447/2880] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:25:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:44.235278 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5210305-f089-4025-b1e9-de371741a534 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:44.236585 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c5210305-f089-4025-b1e9-de371741a534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:44.296592 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5210305-f089-4025-b1e9-de371741a534 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:44.300435 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769b93f-4ef8-4a40-b3fe-59a0532437d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:44.301093 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769b93f-4ef8-4a40-b3fe-59a0532437d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:44.301375 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769b93f-4ef8-4a40-b3fe-59a0532437d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:44.301652 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769b93f-4ef8-4a40-b3fe-59a0532437d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:44.310979 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769b93f-4ef8-4a40-b3fe-59a0532437d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:44.312421 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769b93f-4ef8-4a40-b3fe-59a0532437d7 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:44.322242 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1434/2881] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:44.641626 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca95592-48f7-43f4-b0fa-394ffbb9b61b None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:44.643141 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-5ca95592-48f7-43f4-b0fa-394ffbb9b61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:44.656048 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca95592-48f7-43f4-b0fa-394ffbb9b61b None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:44.671730 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67349cf9-f4d3-4c46-bd94-b8d9c27a3345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:44.673321 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67349cf9-f4d3-4c46-bd94-b8d9c27a3345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:44.673321 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67349cf9-f4d3-4c46-bd94-b8d9c27a3345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:44.673524 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67349cf9-f4d3-4c46-bd94-b8d9c27a3345 admin admin] PATH_INFO: `/v3/projects/71b592d6be0b4f9a9bf9c884bc21097b` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:44.675181 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67349cf9-f4d3-4c46-bd94-b8d9c27a3345 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71b592d6be0b4f9a9bf9c884bc21097b)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:44.677380 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67349cf9-f4d3-4c46-bd94-b8d9c27a3345 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:44.733999 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-67349cf9-f4d3-4c46-bd94-b8d9c27a3345 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71b592d6be0b4f9a9bf9c884bc21097b', 'request_id': 'req-67349cf9-f4d3-4c46-bd94-b8d9c27a3345'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:44.744458 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-67349cf9-f4d3-4c46-bd94-b8d9c27a3345 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71b592d6be0b4f9a9bf9c884bc21097b', 'request_id': 'req-67349cf9-f4d3-4c46-bd94-b8d9c27a3345'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:44.746820 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:44.754091 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-67349cf9-f4d3-4c46-bd94-b8d9c27a3345 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71b592d6be0b4f9a9bf9c884bc21097b', 'request_id': 'req-67349cf9-f4d3-4c46-bd94-b8d9c27a3345'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:44.768691 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-67349cf9-f4d3-4c46-bd94-b8d9c27a3345 admin admin] The token cache is being invalidate because project 71b592d6be0b4f9a9bf9c884bc21097b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:25:44.768979 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-67349cf9-f4d3-4c46-bd94-b8d9c27a3345 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:44.769647 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-67349cf9-f4d3-4c46-bd94-b8d9c27a3345 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:44.771254 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1448/2882] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:25:44 2022] DELETE /identity/v3/projects/71b592d6be0b4f9a9bf9c884bc21097b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:54.317861 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:DELETE=2,keystone:SELECT=33,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:54.766435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:DELETE=1,keystone:SELECT=4 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:03.034739 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769b93f-4ef8-4a40-b3fe-59a0532437d7 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:03.036249 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c769b93f-4ef8-4a40-b3fe-59a0532437d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:03.047927 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769b93f-4ef8-4a40-b3fe-59a0532437d7 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:03.098862 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13207c5d-31c8-4df6-a0f3-cdb09ae77d2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:03.099526 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13207c5d-31c8-4df6-a0f3-cdb09ae77d2a service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:03.099811 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13207c5d-31c8-4df6-a0f3-cdb09ae77d2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:03.100090 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13207c5d-31c8-4df6-a0f3-cdb09ae77d2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:03.102061 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-13207c5d-31c8-4df6-a0f3-cdb09ae77d2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:03.168311 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13207c5d-31c8-4df6-a0f3-cdb09ae77d2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:03.169742 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13207c5d-31c8-4df6-a0f3-cdb09ae77d2a service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:03.179355 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1435/2883] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:26:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:26:09.360157 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67349cf9-f4d3-4c46-bd94-b8d9c27a3345 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:09.362675 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-67349cf9-f4d3-4c46-bd94-b8d9c27a3345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:09.466927 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13207c5d-31c8-4df6-a0f3-cdb09ae77d2a service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:09.469247 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-13207c5d-31c8-4df6-a0f3-cdb09ae77d2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:09.487644 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67349cf9-f4d3-4c46-bd94-b8d9c27a3345 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:09.494610 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0640a-9cbb-4fb6-937b-0756f72f13a7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36b63614297346e29ff276eab444fe5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:09.495624 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0640a-9cbb-4fb6-937b-0756f72f13a7 service placement] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:09.496097 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0640a-9cbb-4fb6-937b-0756f72f13a7 service placement] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:09.496581 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0640a-9cbb-4fb6-937b-0756f72f13a7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:09.497459 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13207c5d-31c8-4df6-a0f3-cdb09ae77d2a service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:09.499896 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-dad0640a-9cbb-4fb6-937b-0756f72f13a7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:09.505452 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941068bc-9a74-4cab-b642-d5080c822698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:09.507018 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941068bc-9a74-4cab-b642-d5080c822698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:09.507452 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941068bc-9a74-4cab-b642-d5080c822698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:09.507872 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941068bc-9a74-4cab-b642-d5080c822698 admin admin] PATH_INFO: `/v3/users/080c7129a52e49c698cb2dc4afe14de7` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:09.509943 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941068bc-9a74-4cab-b642-d5080c822698 admin admin] RBAC: Authorizing `identity:delete_user(user_id=080c7129a52e49c698cb2dc4afe14de7)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:09.513120 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941068bc-9a74-4cab-b642-d5080c822698 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:09.534496 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0640a-9cbb-4fb6-937b-0756f72f13a7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:09.536415 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0640a-9cbb-4fb6-937b-0756f72f13a7 service placement] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:09.558323 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1449/2884] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:26:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:26:09.587271 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-941068bc-9a74-4cab-b642-d5080c822698 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '080c7129a52e49c698cb2dc4afe14de7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:09.600310 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-941068bc-9a74-4cab-b642-d5080c822698 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '080c7129a52e49c698cb2dc4afe14de7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:09.608920 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-941068bc-9a74-4cab-b642-d5080c822698 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '080c7129a52e49c698cb2dc4afe14de7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:09.617302 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-941068bc-9a74-4cab-b642-d5080c822698 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '080c7129a52e49c698cb2dc4afe14de7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:09.617975 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-941068bc-9a74-4cab-b642-d5080c822698 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '080c7129a52e49c698cb2dc4afe14de7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:09.624928 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1436/2885] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:26:09 2022] DELETE /identity/v3/users/080c7129a52e49c698cb2dc4afe14de7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:26:09.807602 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0640a-9cbb-4fb6-937b-0756f72f13a7 service placement] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:09.833694 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0640a-9cbb-4fb6-937b-0756f72f13a7 service placement] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:09.894937 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deff649-4444-4e73-8016-2bb141377014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:09.895712 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deff649-4444-4e73-8016-2bb141377014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:09.895993 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deff649-4444-4e73-8016-2bb141377014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:09.896268 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deff649-4444-4e73-8016-2bb141377014 admin admin] PATH_INFO: `/v3/projects/bd7ec755d96f4dee964a003d6c269c4e` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:09.897465 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deff649-4444-4e73-8016-2bb141377014 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd7ec755d96f4dee964a003d6c269c4e)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:09.899428 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deff649-4444-4e73-8016-2bb141377014 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:09.947037 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6deff649-4444-4e73-8016-2bb141377014 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd7ec755d96f4dee964a003d6c269c4e', 'request_id': 'req-6deff649-4444-4e73-8016-2bb141377014'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:09.953270 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6deff649-4444-4e73-8016-2bb141377014 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd7ec755d96f4dee964a003d6c269c4e', 'request_id': 'req-6deff649-4444-4e73-8016-2bb141377014'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:09.961269 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6deff649-4444-4e73-8016-2bb141377014 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd7ec755d96f4dee964a003d6c269c4e', 'request_id': 'req-6deff649-4444-4e73-8016-2bb141377014'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:09.973109 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6deff649-4444-4e73-8016-2bb141377014 admin admin] The token cache is being invalidate because project bd7ec755d96f4dee964a003d6c269c4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:26:09.973422 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6deff649-4444-4e73-8016-2bb141377014 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:09.974102 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6deff649-4444-4e73-8016-2bb141377014 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:09.975663 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1450/2886] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:26:09 2022] DELETE /identity/v3/projects/bd7ec755d96f4dee964a003d6c269c4e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:26:09.979609 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941068bc-9a74-4cab-b642-d5080c822698 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:09.981050 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-941068bc-9a74-4cab-b642-d5080c822698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:09.994506 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941068bc-9a74-4cab-b642-d5080c822698 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:10.012960 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:10.057728 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1605e0-bb18-4f4e-b7d8-517e9c2acbb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:10.058667 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1605e0-bb18-4f4e-b7d8-517e9c2acbb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:10.058953 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1605e0-bb18-4f4e-b7d8-517e9c2acbb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:10.059231 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1605e0-bb18-4f4e-b7d8-517e9c2acbb3 admin admin] PATH_INFO: `/v3/users/2eeb3db00b9d4c3ba667bd25abe72b27` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:10.060448 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1605e0-bb18-4f4e-b7d8-517e9c2acbb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eeb3db00b9d4c3ba667bd25abe72b27)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:10.063691 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1605e0-bb18-4f4e-b7d8-517e9c2acbb3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:10.111763 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-bd1605e0-bb18-4f4e-b7d8-517e9c2acbb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eeb3db00b9d4c3ba667bd25abe72b27'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:10.125648 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-bd1605e0-bb18-4f4e-b7d8-517e9c2acbb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eeb3db00b9d4c3ba667bd25abe72b27'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:10.132889 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-bd1605e0-bb18-4f4e-b7d8-517e9c2acbb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eeb3db00b9d4c3ba667bd25abe72b27'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:10.141490 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-bd1605e0-bb18-4f4e-b7d8-517e9c2acbb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eeb3db00b9d4c3ba667bd25abe72b27'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:10.142306 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-bd1605e0-bb18-4f4e-b7d8-517e9c2acbb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eeb3db00b9d4c3ba667bd25abe72b27'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:10.149142 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1437/2887] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:26:09 2022] DELETE /identity/v3/users/2eeb3db00b9d4c3ba667bd25abe72b27 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:26:10.331217 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deff649-4444-4e73-8016-2bb141377014 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:10.332846 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-6deff649-4444-4e73-8016-2bb141377014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:10.350083 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deff649-4444-4e73-8016-2bb141377014 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:10.369438 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:10.414736 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383b3a3-84ec-4896-b74e-df990d2b19f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:10.415539 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383b3a3-84ec-4896-b74e-df990d2b19f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:10.415829 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383b3a3-84ec-4896-b74e-df990d2b19f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:10.416111 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383b3a3-84ec-4896-b74e-df990d2b19f0 admin admin] PATH_INFO: `/v3/projects/1ddadfa11c1c4dbd8b2dacc6ab97f722` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:10.417341 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383b3a3-84ec-4896-b74e-df990d2b19f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ddadfa11c1c4dbd8b2dacc6ab97f722)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:10.419350 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383b3a3-84ec-4896-b74e-df990d2b19f0 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:10.468150 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-f383b3a3-84ec-4896-b74e-df990d2b19f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ddadfa11c1c4dbd8b2dacc6ab97f722', 'request_id': 'req-f383b3a3-84ec-4896-b74e-df990d2b19f0'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:10.473774 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-f383b3a3-84ec-4896-b74e-df990d2b19f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ddadfa11c1c4dbd8b2dacc6ab97f722', 'request_id': 'req-f383b3a3-84ec-4896-b74e-df990d2b19f0'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:10.481297 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-f383b3a3-84ec-4896-b74e-df990d2b19f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ddadfa11c1c4dbd8b2dacc6ab97f722', 'request_id': 'req-f383b3a3-84ec-4896-b74e-df990d2b19f0'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:10.493682 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-f383b3a3-84ec-4896-b74e-df990d2b19f0 admin admin] The token cache is being invalidate because project 1ddadfa11c1c4dbd8b2dacc6ab97f722 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:26:10.493998 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-f383b3a3-84ec-4896-b74e-df990d2b19f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:10.494669 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-f383b3a3-84ec-4896-b74e-df990d2b19f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:10.496223 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1451/2888] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:26:10 2022] DELETE /identity/v3/projects/1ddadfa11c1c4dbd8b2dacc6ab97f722 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:26:17.544069 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1605e0-bb18-4f4e-b7d8-517e9c2acbb3 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:17.545527 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bd1605e0-bb18-4f4e-b7d8-517e9c2acbb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:17.612855 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1605e0-bb18-4f4e-b7d8-517e9c2acbb3 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:17.669307 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed92c13-8962-47ab-bec5-80e25388e899 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69b669a5ba4e4294baeea369949e3e6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:17.670005 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed92c13-8962-47ab-bec5-80e25388e899 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:17.670298 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed92c13-8962-47ab-bec5-80e25388e899 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:17.670581 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed92c13-8962-47ab-bec5-80e25388e899 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:17.672500 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3ed92c13-8962-47ab-bec5-80e25388e899 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:17.692156 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:17.748352 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed92c13-8962-47ab-bec5-80e25388e899 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:17.750483 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed92c13-8962-47ab-bec5-80e25388e899 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:17.761074 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1438/2889] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:26:17 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:26:20.490820 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:21.026864 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c542b5-d92d-44c3-8a5c-b6a66eca996d None None] REQUEST_METHOD: `POST` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:21.027154 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c542b5-d92d-44c3-8a5c-b6a66eca996d None None] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:21.027435 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c542b5-d92d-44c3-8a5c-b6a66eca996d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:21.045711 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.auth.core [None req-b3c542b5-d92d-44c3-8a5c-b6a66eca996d None None] MFA Rules not processed for user `373d58e9f0e6472dad412ac02bff2b94`. Rule list: `[]` (Enabled: `True`). {{(pid=80439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:21.046867 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-b3c542b5-d92d-44c3-8a5c-b6a66eca996d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:21.054660 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1452/2890] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 16:26:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:26:27.756155 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=22 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:31.040324 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=3 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:58.386751 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed92c13-8962-47ab-bec5-80e25388e899 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:58.388191 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3ed92c13-8962-47ab-bec5-80e25388e899 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:58.446415 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed92c13-8962-47ab-bec5-80e25388e899 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:58.450749 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234cfa8f-b671-4dbb-954f-8353ab97c237 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:58.451428 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cfa8f-b671-4dbb-954f-8353ab97c237 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:58.451710 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cfa8f-b671-4dbb-954f-8353ab97c237 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:58.451986 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cfa8f-b671-4dbb-954f-8353ab97c237 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:58.453782 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-234cfa8f-b671-4dbb-954f-8353ab97c237 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:58.464096 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234cfa8f-b671-4dbb-954f-8353ab97c237 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:58.466632 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234cfa8f-b671-4dbb-954f-8353ab97c237 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:58.474982 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1439/2891] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:27:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:28:08.471160 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=26 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:19.626889 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c542b5-d92d-44c3-8a5c-b6a66eca996d None None] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:19.638131 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c542b5-d92d-44c3-8a5c-b6a66eca996d None None] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:19.642077 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17cdd22-9dd8-4862-9f06-2f5c28b5e16b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:19.642757 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cdd22-9dd8-4862-9f06-2f5c28b5e16b service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:19.643130 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cdd22-9dd8-4862-9f06-2f5c28b5e16b service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:19.643324 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cdd22-9dd8-4862-9f06-2f5c28b5e16b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:19.646178 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-a17cdd22-9dd8-4862-9f06-2f5c28b5e16b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:19.756451 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17cdd22-9dd8-4862-9f06-2f5c28b5e16b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:19.757885 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17cdd22-9dd8-4862-9f06-2f5c28b5e16b service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:19.781979 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1453/2892] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:28:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:28:20.988659 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234cfa8f-b671-4dbb-954f-8353ab97c237 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:20.990265 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-234cfa8f-b671-4dbb-954f-8353ab97c237 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:21.005339 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234cfa8f-b671-4dbb-954f-8353ab97c237 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:21.009260 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9915e6-8988-4ecf-8770-54b52fc22263 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:21.009951 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9915e6-8988-4ecf-8770-54b52fc22263 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:21.010256 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9915e6-8988-4ecf-8770-54b52fc22263 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:21.010536 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9915e6-8988-4ecf-8770-54b52fc22263 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:21.012352 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0c9915e6-8988-4ecf-8770-54b52fc22263 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:21.029711 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9915e6-8988-4ecf-8770-54b52fc22263 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:21.031327 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9915e6-8988-4ecf-8770-54b52fc22263 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:21.045094 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1440/2893] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:28:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:28:29.778689 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=43 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:31.038334 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=6 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:32.302805 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17cdd22-9dd8-4862-9f06-2f5c28b5e16b service neutron] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:32.311830 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17cdd22-9dd8-4862-9f06-2f5c28b5e16b service neutron] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:32.315473 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a03f7-5aa1-40c6-94fb-390a31a12fec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a84f8aed50c4b3e8e5b33c5b587b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:32.316281 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a03f7-5aa1-40c6-94fb-390a31a12fec service neutron] REQUEST_METHOD: `GET` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:32.316567 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a03f7-5aa1-40c6-94fb-390a31a12fec service neutron] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:32.316846 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a03f7-5aa1-40c6-94fb-390a31a12fec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:32.318665 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-eb3a03f7-5aa1-40c6-94fb-390a31a12fec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:32.384629 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a03f7-5aa1-40c6-94fb-390a31a12fec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:32.386209 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a03f7-5aa1-40c6-94fb-390a31a12fec service neutron] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:32.396764 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1454/2894] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:28:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:28:38.703877 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9915e6-8988-4ecf-8770-54b52fc22263 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:38.705562 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0c9915e6-8988-4ecf-8770-54b52fc22263 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:38.727838 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9915e6-8988-4ecf-8770-54b52fc22263 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:38.732808 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d5b0a-8d3b-47cd-8b6e-03e911e1bca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:38.733683 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d5b0a-8d3b-47cd-8b6e-03e911e1bca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:38.734105 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d5b0a-8d3b-47cd-8b6e-03e911e1bca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:38.734392 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d5b0a-8d3b-47cd-8b6e-03e911e1bca0 admin admin] PATH_INFO: `/v3/users/373d58e9f0e6472dad412ac02bff2b94` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:38.735592 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395d5b0a-8d3b-47cd-8b6e-03e911e1bca0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=373d58e9f0e6472dad412ac02bff2b94)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:38.737564 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395d5b0a-8d3b-47cd-8b6e-03e911e1bca0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:38.788227 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-395d5b0a-8d3b-47cd-8b6e-03e911e1bca0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '373d58e9f0e6472dad412ac02bff2b94'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:28:38.801651 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-395d5b0a-8d3b-47cd-8b6e-03e911e1bca0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '373d58e9f0e6472dad412ac02bff2b94'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:28:38.808929 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-395d5b0a-8d3b-47cd-8b6e-03e911e1bca0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '373d58e9f0e6472dad412ac02bff2b94'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:28:38.816459 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-395d5b0a-8d3b-47cd-8b6e-03e911e1bca0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '373d58e9f0e6472dad412ac02bff2b94'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:28:38.817100 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-395d5b0a-8d3b-47cd-8b6e-03e911e1bca0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '373d58e9f0e6472dad412ac02bff2b94'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:28:38.823213 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1441/2895] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:28:38 2022] DELETE /identity/v3/users/373d58e9f0e6472dad412ac02bff2b94 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:28:39.008839 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a03f7-5aa1-40c6-94fb-390a31a12fec service neutron] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:39.020660 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a03f7-5aa1-40c6-94fb-390a31a12fec service neutron] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:39.085598 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fe7be-99bd-4ecc-8488-8f2a949dae16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:39.086507 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fe7be-99bd-4ecc-8488-8f2a949dae16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:39.086798 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fe7be-99bd-4ecc-8488-8f2a949dae16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:39.087174 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fe7be-99bd-4ecc-8488-8f2a949dae16 admin admin] PATH_INFO: `/v3/projects/b77126371c5d44db8ab75fbfca0737af` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:39.088395 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1fe7be-99bd-4ecc-8488-8f2a949dae16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b77126371c5d44db8ab75fbfca0737af)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:39.090390 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1fe7be-99bd-4ecc-8488-8f2a949dae16 admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:39.138358 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6c1fe7be-99bd-4ecc-8488-8f2a949dae16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b77126371c5d44db8ab75fbfca0737af', 'request_id': 'req-6c1fe7be-99bd-4ecc-8488-8f2a949dae16'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:28:39.143918 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6c1fe7be-99bd-4ecc-8488-8f2a949dae16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b77126371c5d44db8ab75fbfca0737af', 'request_id': 'req-6c1fe7be-99bd-4ecc-8488-8f2a949dae16'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:28:39.154734 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6c1fe7be-99bd-4ecc-8488-8f2a949dae16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b77126371c5d44db8ab75fbfca0737af', 'request_id': 'req-6c1fe7be-99bd-4ecc-8488-8f2a949dae16'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:28:39.166938 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6c1fe7be-99bd-4ecc-8488-8f2a949dae16 admin admin] The token cache is being invalidate because project b77126371c5d44db8ab75fbfca0737af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:28:39.166938 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6c1fe7be-99bd-4ecc-8488-8f2a949dae16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:28:39.168282 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-6c1fe7be-99bd-4ecc-8488-8f2a949dae16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:28:39.170128 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1455/2896] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:28:39 2022] DELETE /identity/v3/projects/b77126371c5d44db8ab75fbfca0737af => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:28:39.174157 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d5b0a-8d3b-47cd-8b6e-03e911e1bca0 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:39.176909 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-395d5b0a-8d3b-47cd-8b6e-03e911e1bca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:39.192460 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d5b0a-8d3b-47cd-8b6e-03e911e1bca0 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:39.264801 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd670483-ee06-49c3-bf3b-b775ef3ff7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:39.265679 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd670483-ee06-49c3-bf3b-b775ef3ff7c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:39.265987 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd670483-ee06-49c3-bf3b-b775ef3ff7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:39.266265 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd670483-ee06-49c3-bf3b-b775ef3ff7c0 admin admin] PATH_INFO: `/v3/users/f8a60b97a4bb4749a145304edb0b07cf` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:39.267447 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd670483-ee06-49c3-bf3b-b775ef3ff7c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8a60b97a4bb4749a145304edb0b07cf)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:39.270078 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd670483-ee06-49c3-bf3b-b775ef3ff7c0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:39.320364 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-bd670483-ee06-49c3-bf3b-b775ef3ff7c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8a60b97a4bb4749a145304edb0b07cf'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:28:39.344940 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-bd670483-ee06-49c3-bf3b-b775ef3ff7c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8a60b97a4bb4749a145304edb0b07cf'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:28:39.352276 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-bd670483-ee06-49c3-bf3b-b775ef3ff7c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8a60b97a4bb4749a145304edb0b07cf'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:28:39.354665 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:39.363523 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-bd670483-ee06-49c3-bf3b-b775ef3ff7c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8a60b97a4bb4749a145304edb0b07cf'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:28:39.364580 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-bd670483-ee06-49c3-bf3b-b775ef3ff7c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8a60b97a4bb4749a145304edb0b07cf'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:28:39.371699 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1442/2897] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 16:28:39 2022] DELETE /identity/v3/users/f8a60b97a4bb4749a145304edb0b07cf => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:28:39.555337 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fe7be-99bd-4ecc-8488-8f2a949dae16 admin admin] Authenticating user token {{(pid=80439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:39.557042 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.fernet_utils [None req-6c1fe7be-99bd-4ecc-8488-8f2a949dae16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:39.571630 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fe7be-99bd-4ecc-8488-8f2a949dae16 admin admin] Validating token access rules against request {{(pid=80439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:39.594364 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:39.642623 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea3aaa-46ab-45d5-a1bb-6ef91df3e2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9593d0f94eed4e3e83f1323aab2716a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47ead53fd5cc424c8de9383e832d40a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:39.643591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea3aaa-46ab-45d5-a1bb-6ef91df3e2cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:39.644307 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea3aaa-46ab-45d5-a1bb-6ef91df3e2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:39.644307 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea3aaa-46ab-45d5-a1bb-6ef91df3e2cc admin admin] PATH_INFO: `/v3/projects/4be9ac41669a4f3194ef165f3fc5cb31` {{(pid=80439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:39.645438 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea3aaa-46ab-45d5-a1bb-6ef91df3e2cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4be9ac41669a4f3194ef165f3fc5cb31)` {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:39.647407 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea3aaa-46ab-45d5-a1bb-6ef91df3e2cc admin admin] RBAC: Authorization granted {{(pid=80439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:39.698931 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-4bea3aaa-46ab-45d5-a1bb-6ef91df3e2cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4be9ac41669a4f3194ef165f3fc5cb31', 'request_id': 'req-4bea3aaa-46ab-45d5-a1bb-6ef91df3e2cc'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:28:39.712297 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-4bea3aaa-46ab-45d5-a1bb-6ef91df3e2cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4be9ac41669a4f3194ef165f3fc5cb31', 'request_id': 'req-4bea3aaa-46ab-45d5-a1bb-6ef91df3e2cc'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:28:39.720373 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-4bea3aaa-46ab-45d5-a1bb-6ef91df3e2cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4be9ac41669a4f3194ef165f3fc5cb31', 'request_id': 'req-4bea3aaa-46ab-45d5-a1bb-6ef91df3e2cc'} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:28:39.733070 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-4bea3aaa-46ab-45d5-a1bb-6ef91df3e2cc admin admin] The token cache is being invalidate because project 4be9ac41669a4f3194ef165f3fc5cb31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:28:39.733357 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-4bea3aaa-46ab-45d5-a1bb-6ef91df3e2cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:28:39.734032 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG keystone.notifications [None req-4bea3aaa-46ab-45d5-a1bb-6ef91df3e2cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:28:39.735591 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: [pid: 80439|app: 0|req: 1456/2898] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 16:28:39 2022] DELETE /identity/v3/projects/4be9ac41669a4f3194ef165f3fc5cb31 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:28:49.368105 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:49.729954 dv-cinder-837167-3-iscsi devstack@keystone.service[80439]: DEBUG dbcounter [-] [80439] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:48.682436 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd670483-ee06-49c3-bf3b-b775ef3ff7c0 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:48.684268 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bd670483-ee06-49c3-bf3b-b775ef3ff7c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:48.762186 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd670483-ee06-49c3-bf3b-b775ef3ff7c0 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:48.827552 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49c9fd-bc9b-47aa-baad-4899a54ff894 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36b63614297346e29ff276eab444fe5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1177a85a0de4276898daf4a24ff069f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:48.828213 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49c9fd-bc9b-47aa-baad-4899a54ff894 service placement] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:48.828496 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49c9fd-bc9b-47aa-baad-4899a54ff894 service placement] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:48.828771 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49c9fd-bc9b-47aa-baad-4899a54ff894 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:48.832626 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ac49c9fd-bc9b-47aa-baad-4899a54ff894 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:48.844746 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49c9fd-bc9b-47aa-baad-4899a54ff894 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:48.846836 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49c9fd-bc9b-47aa-baad-4899a54ff894 service placement] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:48.857356 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1443/2899] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 16:30:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:30:58.852807 dv-cinder-837167-3-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=58 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}